The Trice Insert Algorithm

Contents

---

• 1. Starting Conditions
• 2. Aims
• 3. Method
  • 3.1. Trice Insert Initialization
• 4. User Code Patching (trice insert)
• 5. User Code Patching Examples
• 6. Exclude folders & files from being parsed (pull request 529)
• 7. ID Usage Options
• 8. General ID Management Information
  • 8.1. Option Cleaning in a Post-build process
  • 8.2. Option Let the inserted Trice ID be a Part of the User Code
  • 8.3. Option Cleaning on Repository Check-In

---

1. Starting Conditions

@@ To understand this chapter you should look into the Trice tool source code. @@

• Before trice i is executed on a source tree, the starting conditions are partially undefined:
  • A trice ID list file til.json file must exist, but it is allowed to be empty.
    • The til.json is a serialized key-value map, where
      • the keys are the IDs i and
      • the values are Trice format string structs (bit width plus format string) named f.
      • When de-serializing, it is not impossible, that an ID is used more than one times. This can only happen, when til.json was edited manually, what normally is not done. But could be the result of a git merge.
        • The trice tool will report that as error and stop. The user then has to correct the error manually, for example by deleting one of the doubled keys.
      • This ID look-up is the key-value map idToFmt TriceIDLookUp as map[TriceID]TriceFmt.
        • Each ID i as key, points to one and only one f.
        • The TriceFmt structs contains the parameter width and the format string.
      • The idToFmt is reverted then into fmtToId triceFmtLookUp as map[TriceFmt]TriceIDs.
        • TriceIDs is a triceID slice because the identical f can have several ids (no shared IDs).
        • The format struct f look-up map fmtToId is used internally for faster access and always in sync with idToFmt.
    • idToFmt and fmtToId together are named lu.
  • A location information file li.json may exist or not.
    • The li.json is a serialized key-value map idToLocRef TriceIDLookUpLI, a map[TriceID]TriceLI, where
      • the keys are the IDs i and
      • the values are the location information (filename, line and position in line) structs.
    • Each ID as key points to one and only one location information.
• The til.json IDs may occur in the source tree not at all, once or several times. Also it is not guarantied, that the source tree Trices match the til.json value.
  • That is possible after code edit, for example or code copied or modified.
  • One and only one position is used and relevant, all others are ignored. If no til.json exists on the expected location the user must provide one, at least an empty file.
• The li.json IDs may occur in the source tree not at all, once or several times. Also it is not guarantied, that the source tree Trices match the li.json value.
  • One and only one position is used and relevant, all others are ignored. If no li.json exists on the expected location trice insert creates one there.
• The src tree can contain IDs not present inside til.json. This state is seldom, for example after adding sources containing IDs.

2. Aims

• The trice insert main aim is to have a consistent state between til.json, li.json and the source tree with no ID used twice.
• Also the changes should be minimal.
• As a general rule lu is only extendable.
• LI is updated for selected sites while ownership outside the scan is retained.
• Files are read in parallel. Allocation and the final assignment are planned in source order before source publication.
• The final assignment follows the identical-Trice ordering rule across the selected files. A rename may require a new ID when the old LI path lies outside that selection.

3. Method

3.1. Trice Insert Initialization

// insertIDsData holds the insert run specific data.
type insertIDsData struct {
    idToFmt    TriceIDLookUp     // idToFmt is a trice ID lookup map and is generated from existing til.json file at the begin of SubCmdIdInsert. This map is only extended during SubCmdIdInsert and goes back into til.json afterwards.
    fmtToId    triceFmtLookUp    // fmtToId is a trice fmt lookup map (reversed idToFmt for faster operation) and kept in sync with idToFmt. Each fmt can have several trice IDs (slice).
    idToLocRef TriceIDLookUpLI   // idToLocInf is the trice ID location information as reference generated from li.json (if exists) at the begin of SubCmdIdInsert and is not modified at all. At the end of SubCmdIdInsert a new li.json is generated from itemToId.
    itemToId   TriceItemLookUpID // itemToId is a trice item lookup ID map, extended from source tree during SubCmdIdInsert after each found and maybe modified trice item.
    idToItem   TriceIDLookupItem // idToItem is a trice ID lookup item map (reversed itemToId for faster operation) and kept in sync with itemToId.
}

• Create an insertIDsData instance.
• De-serialize til.json into idToFmt and fmtToId. On error abort and report for manual correction. One result is a slice with used IDs.
• De-serialize li.json into idToLocRef. On error abort and report for manual correction. As result the slice with used IDs is extended.
  • If li.json contains IDs not already inside til.json, these are reported as warning.
  • idToLocRef stays untouched and is used only in cases when identical f are found.
• Create a slice IDSpace with numbers IDMin … IDMax (1 … 16383, or 1000 … 1999 if specified in the command line that way)
• Remove all used IDs from IDSpace.
  • If used IDs outside IDMin and IDmax, for example IDMin=1000, IDmax=1999 and some used IDs are bigger or smaller these are not removable from IDroom what is ok.
• Create empty itemToId and idToItem.
• Walk the src and create a source tree map STM with
  • key=Trice+LI and
  • value=ID.
• During STM creation use these rules:
  • If the next found f src ID == n != 0:
    • If ID n already inside STM set ID = 0 (that is brutal but ok)
    • Otherwise extend STM with ID n and remove n from ID space
      • It is possible, f is used n times with different IDs, so that is no problem.
      • If f occurs several times with the same ID, the first occurrence provisionally claims it. Final group ordering can assign that ID to another site.
  • If the next found f src ID == 0 (normal case after trice z):
    • Look in flu
      • If not there, create new id and extend STM.
        • The new ID is “new”, so forbidden to be inside ilu.
        • If it is accidentally somewhere in the so far unparsed src, we do not know that and therefore do not care about.
          • That is a seldom case and not worth to parse the source tree twice all the time.
        • Patch id into source and extend STM.
      • If the ID slice has len 1 (usually the case), take that n, extend STM and remove f from flu.
        • That is important because f could be copied before.
      • If the ID slice has a len > 1 (several IDs on the same string) check li
        • If li is empty, just remove the first id from the slice and extend STM
        • Loop over slice IDs
          • If a file matches, take the first occurrence, extend STM and remove id from the ID slice
          • If no file matches do the same as when li is empty.
          • That means, after file renaming or code copying between files during trice z state, new IDs are generated for that parts.
            • That is only for same f with several IDs cases
          • File changes during trice i state are ok, because STM is generated with the IDs inside the sources.

Until here the algorithm seem to be ok.

• STM is not needed but maybe helpful during debugging.

• STM than is usable to regenerate li.json and to extend til.json

• After candidate selection, sort each identical group’s selected eligible IDs numerically and its sites by relative path, line, and column; then publish matching sources and LI. The Insert cache must contain that final permutation too.
• An unchanged Clean/Insert cycle with the same selected ID pool restores the same assignment. Editing the group of identical sites may change any of their IDs according to the ordering rule.

4. User Code Patching (trice insert)

• A Trice ID is inserted by trice insert as shown in the table:

  Unpatched User Code	After trice insert	Remark
  trice( "Hi!\n");	trice( iD(12345), "Hi!\n");	no stamps
  Trice( "Hi!\n");	Trice( iD(12345), "Hi!\n");	16-bit stamps
  TRice( "Hi!\n");	TRice( iD(12345), "Hi!\n");	32-bit stamps

• trice insert preserves existing whitespace directly after the opening Trice parenthesis. For example, trice("Hi!\n"); becomes trice(iD(12345), "Hi!\n");, while trice( "Hi!\n"); becomes trice( iD(12345), "Hi!\n");. This keeps trice insert and trice clean formatting-preserving for common formatter styles.

• The -w or -spaceInsideParenthesis switch still requests wide generated ID formatting, for example trice( iD( 12345 ), "Hi!\n");. If whitespace after ( already exists, it is preserved instead of being normalized.

• Legacy code is handled this way:

  Unpatched User Code	After trice insert	Remark
  TRICE( "Hi!\n");	TRICE( id(12345), "Hi!\n");	no stamps after trice i -defaultStampSize 0
  TRICE( "Hi!\n");	TRICE( Id(12345), "Hi!\n");	16-bit stamps after trice i -defaultStampSize 16
  TRICE( "Hi!\n");	TRICE( ID(12345), "Hi!\n");	32-bit stamps after trice i -defaultStampSize 32
  TRICE( id(0), "Hi!\n");	TRICE( id(12345), "Hi!\n");	no stamps
  TRICE( Id(0), "Hi!\n");	TRICE( Id(12345), "Hi!\n");	16-bit stamps
  TRICE( ID(0), "Hi!\n");	TRICE( ID(12345), "Hi!\n");	32-bit stamps

• A pre-build step trice insert generates the Id(12345) part. Examples:

  • trice i in your project root expects a til.json file there and checks sources and til.json for changes to insert.
  • trice i -v -i ../../../til.json -src ../src -src ../lib/src -src ./ is a typical case as automated pre-build step in your project settings telling Trice to scan the project dir and two external directories. Even trice i is fast, it is generally quicker to search only relevant places.

5. User Code Patching Examples

• A Trice ID is modified as shown in these cases:

  • Previously inserted (patched) user code copied to a different location:

    trice(iD(12345), "Hi!\n"); // copied
    trice(iD(12345), "Hi!\n"); // original
    trice(iD(12345), "Hi!\n"); // copied
    

  • After updating (patching) again:

    trice(iD(12345), "Hi!\n");
    trice(iD( 1233), "Hi!\n"); // re-patched
    trice(iD( 1234), "Hi!\n"); // re-patched
    

    • If the code is copied inside the same file, the first occurrence after the copy stays unchanged and the following are modified.
    • If the code is copied to other files only, the copies get new IDs.

  • Previously inserted (patched) user code copied and modified:

    trice(iD(12345), "Ha!\n"); // copied and modified
    trice(iD(12345), "Hi!\n"); // original
    trice(iD(12345), "Ha!\n"); // copied and modified
    

  • After updating (patching) again:

    trice(iD( 2333), "Ha!\n"); // re-patched
    trice(iD(12345), "Hi!\n"); // unchanged
    trice(iD( 1234), "Ha!\n"); // re-patched
    

  • If the code is copied to other files, it is re-patched.

• A Trice ID is stays the same if the stamp size is changed. Example:

  trice( iD(12345), "Hi!" ); // original
  

  TRice( iD(12345), "Hi!" ); // manually changed stamp size and then "trice i" performed.
  

6. Exclude folders & files from being parsed (pull request 529)

The pull request #529 introduces key enhancement:

    -exclude Flag
    Introduces a command-line flag -exclude that allows users to specify one or more source addresses to be omitted from scanning or processing. This improves flexibility in environments with known noisy or irrelevant sources.

-exclude Flag Example

The -exclude flag can be used multiple times to omit specific files or directories from scanning. Wildcards are not supported.

trice insert -v -src ./_test/ -exclude _test/src/trice.h -exclude _test/generated/

7. ID Usage Options

• Per default the trice insert command chooses randomly a so far unused ID for new format strings and extends til.json.
• After trice c all src IDs are removed or 0. In this state the src should go into the version management system.

8. General ID Management Information

• Each format string gets its unique trice ID. If the same format string is used on different source code locations it gets different trice IDs this way allowing a reliable location information.
• The trice ID-instead-of-String idea lives from pre-compile patching of the user code.
• The user has full control how to deal with that.
• There are the 3 following options and the user has to decide which fits best for him. The Trice Cache is probably the best fitting setup for many users.

8.1. Option Cleaning in a Post-build process

• The code is visually free of IDs all the time.

8.2. Option Let the inserted Trice ID be a Part of the User Code

• This is the legacy method. It allows unchanged src translation into code without using the trice tool.
• It is very robust and maybe needed in nasty debugging situations.
• It allows to reconstruct lost til.json information.
• Recommendet for small projects.

8.3. Option Cleaning on Repository Check-In

• The code is visually free of IDs only inside the repository.

(back to top)