Prettify code

Note: Until the release of Igor Pro version 8.05, make sure you have updated to a recent nightly build (version 8.05B01 build 36043 or later) if you want to play with this code. The code posted below exposes a bug that can cause Igor to crash.

Case-corrects Igor function and operation names, and removes trailing whitespace.

Note that the file containing this code will be loaded as an independent module. You'll have to show hidden items in the procedure browser to see the code.

The most recent version of this code can be found in the compilation of text-editing code posted here.

#pragma TextEncoding="UTF-8"
#pragma rtGlobals=3
#pragma IgorVersion=8
#pragma IndependentModule=Prettify
#pragma version=1.5

// https://www.wavemetrics.com/user/tony

// To prettify selected code: cmd-5 (Mac), ctrl-5 (Windows)

// customize prettification here
Strconstant ksIgnoreList="" // function and operation names to ignore
Strconstant ksCustomList="" // add or override items
Constant kMinWordLength=3 // avoiding 2 character words like Pi and ei
// reduces possibility of replacing text in binary represented as ASCII

Menu "Edit"
    "Prettify Clipboard Code", /Q, PutScrapText Prettify#PrettifyCode(GetScrapText())
    "Prettify Selected Code/5", /Q, PrettifySelection()
End

Function PrettifySelection()
   
    String strScrap = GetScrapText()
    PutScrapText ""
    DoIgorMenu "Edit" "Copy"
    PutScrapText PrettifyCode(GetScrapText())
    DoIgorMenu "Edit" "Paste"
    PutScrapText strScrap // leave clipboard state unchanged
End

// Returns a string with known function and operation names case-corrected.
// Commented and quoted text is ignored. Trailing whitespace is eradicated.
// Blocks of code representing proc pictures are ignored, provided that the
// selection starts before the Picture keyword.
Function /S PrettifyCode(String strText)
   
    if(strlen(strText)==0)
        return ""
    endif
   
    String strList = FunctionList("*",";","") + OperationList("*",";","")
    String keyWords="End;EndMacro;EndStructure;Function;Macro;Picture;Proc;Structure;Window;"
    // keyWords+="#if;#endif;#ifdef;#ifndef;#include;#pragma;#undef;" // can't match these as whole words
    keyWords+="Constant;DoPrompt;GalleryGlobal;hide;IgorVersion;IndependentModule;Menu;ModuleName;MultiThread"
    keyWords+="Override;Popup;ProcGlobal;Prompt;root;rtGlobals;Static;Strconstant;Submenu;TextEncoding;ThreadSafe;version;"
    String subtypes="ButtonControl;CameraWindow;CDFFunc;CheckboxControl;CursorStyle;DrawUserShape;FitFunc;"
    subtypes+="GizmoPlot;Graph;GraphMarquee;GraphStyle;GridStyle;Layout;LayoutMarquee;LayoutStyle;ListBoxControl;"
    subtypes+="Panel;PopupMenuControl;SetVariableControl;SliderControl;TabControl;Table;TableStyle;"
    String objectRefs="DFREF;FUNCREF;NVAR;STRUCT;SVAR;WAVE;"
    String flowControl="AbortOnRTE;AbortOnValue;break;case;catch;continue;default;do;else;elseif;endfor;endif;"
    flowControl+="endswitch;endtry;for;if;return;strswitch;switch;try;while;"
    strList+=keyWords+subtypes+objectRefs+flowControl
   
    // remove words in ksIgnoreList, substitute or add words in ksCustomList
    strList = RemoveFromList(ksIgnoreList+ksCustomList, strList, ";", 0) + ksCustomList
    WAVE /T wList = ListToTextWave(strList, ";")
    // remove words with fewer than kMinWordLength characters
    Variable i
    for(i=numpnts(wList)-1; i>=0; i-=1)
        if(strlen(wList[i]) < kMinWordLength)
            DeletePoints i, 1, wList
        endif
    endfor
   
    Variable endsWithReturn = (cmpstr(strText[strlen(strText)-1], "\r") == 0)
    WAVE /T wText = ListToTextWave(strText, "\r")
    Variable lastLine = numpnts(wText) - 1
    wText += SelectString(p<(lastLine) || endsWithReturn, "", "\r")
   
    // create a mask for lines that should not be altered
    Variable numLines=numpnts(wText)
    Make /free/N=(numLines) wIgnore=0
    Grep /Q/INDX/E="(?i)^\s*Picture\b" wText
    WAVE W_Index
    Variable vLine
   
    for (i=0;i<numpnts(W_Index);i+=1)
        vLine=w_Index[i]+1 // the line that starts the Picture binary as ASCII
        if(vLine>numLines)
            break
        endif
        do
            wIgnore[vLine]=1
            vLine+=1
        while(vLine<numLines && GrepString(wText[vLine],"(?i)^\s*End\b")==0)
    endfor

    multithread wText = PrettifyLine(wText, wList, wIgnore)
    wfprintf strText, "%s", wText
    return strText
End

// strLine should include \r if one was present in input text
ThreadSafe Function /S PrettifyLine(String strLine, WAVE /T wordList, Variable ignore)
   
    if(ignore)
        return strLine
    endif
   
    WAVE /B mask = QuoteAndCommentMask(strLine)
    String strCode="", word=""
    int bytes = -1, len=numpnts(mask)
    // find position of last unmasked character in line
    Duplicate /free mask rmask
    Reverse mask /D=rmask
    FindValue /I=0 rmask // find first 0 in reversed mask wave
    if(v_value==-1) // entire line is masked
        strCode=""
    else
        strCode=strLine[0,len-1-v_value]
    endif
    // strCode should contain any code but no commented text

    do // loop through words in line
        SplitString /E="([[:alnum:]]+)(.*)" strCode, word, strCode
        FindValue /TEXT=word/TXOP=4/Z wordList
        if(V_value > -1)
            strLine = ReplaceWord(word, strLine, wordList[V_value], mask)
        endif
    while(strlen(strCode))

    // remove trailing whitespace, ie. spaces and tabs that follow non-whitespace at end of line
    if(GrepString(strLine, "\S[[:blank:]]+\r$"))
        do
            bytes = strlen(strLine)
            strLine = ReplaceString(" \r", strLine, "\r")
            strLine = ReplaceString("\t\r", strLine, "\r")
        while(bytes > strlen(strLine))
    endif
   
    return strLine
End

ThreadSafe Function /WAVE QuoteAndCommentMask(String strLine)
       
    Variable startByte, endByte, len
    len=strlen(strLine)
    Make /B/free/N=(len) mask=0
    startByte=0
    do
        startByte=strsearch(strLine, "\"", startByte)
        if(startByte==-1) // no more quoted text before end of strLine
            break
        endif
        endByte=startByte
        do
            endByte=strsearch(strLine, "\"", endByte+1)
            if(endByte==-1)
                endByte=len-1
                break
            endif
            // endByte is possible end of quote
            // remove escaped backslashes!
            strLine[startByte,endByte]=ReplaceString("\\\\", strLine[startByte,endByte], "  ")
        while(cmpstr(strLine[endByte-1], "\\")==0) // ignore escaped quotes
        // found end of quote
        mask[startByte, endByte]=1
        startByte=endByte+1
    while(1) // look for next quoted text
    // quoted text is now masked
    startByte=0
    do
        startByte=strsearch(strLine, "//", startByte)
        if(startByte==-1) // no comment before end of strLine, we're done
            return mask
        endif
        if(mask[startByte]==1)
            startByte+=1
            continue
        endif
        // startByte is start of comment
        mask[startByte,len-1]=1
        return mask
    while(1)
End

// Wrapper for ReplaceString, replaces whole words
// characters inStr[i] where mask[i]==1 are ignored
ThreadSafe Function /S ReplaceWord(replaceThisWord, inStr, withThisWord, mask)
    String replaceThisWord, inStr, withThisWord
    WAVE /B mask
   
    String substring=""
    Variable startByte=0, endByte=0

    do // loop through matches
        startByte = strsearch(inStr, replaceThisWord, endByte, 2)
        if(startByte == -1)
            return inStr
        endif
        endByte = startByte + strlen(replaceThisWord)
        if(mask[startbyte]==1)
            continue
        endif
        substring = inStr[startByte-1, endByte+1] // will be clipped to 0, strlen(inStr)
        // check that match is whole word
        if(GrepString(substring, "(?i)\\b" + replaceThisWord + "\\b"))
            substring = ReplaceString(replaceThisWord, substring, withThisWord)
        endif
        inStr = inStr[0, startByte-2] + substring + inStr[endByte+2, Inf]
    while(1)
End

 

The updated version posted above ignores quoted text and blocks of code representing proc pictures, provided that the selection starts before the Picture keyword. If I have got this right, Prettify will case-correct only words that are colored by Igor's syntax highlighting as operation or function names.

Great that you have everything in one package now. Feel free to add my align comments code as well, so that we have everything in one neat file.

I posted the file with everything as a new thread in code snippets forum. Any user can edit.

I've played around with it. And it is quite useful! Are you using github for other projects? Collaboration is more easily there IMHO.

I'm posting below a modified version.

Changes:

- Properly bailout on old IP

- Factor out mask creation and ignore window recreation macros as well

- Made hardcoded customization constants into optional parameters

- Added code to prettify all procedure files in a disc folder

- Handle ignoreList without trailing semicolon

- Simplify trailing whitespace removal

- Allow custom eol character

With that I could prettify some 60k LOC project using

Prettify#prettifyfolder("e:projekte:mies-igor:Packages:MIES", eol = "\n", customList = "variable;string;static;threadsafe;StringMatch", ignoreList="graph;proc;panel")

It does still compile and I could not yet spot any errors.
 

#pragma TextEncoding="UTF-8"
#pragma rtGlobals=3
#pragma IgorVersion=8
#pragma IndependentModule=Prettify
#pragma version=1.5

#if (NumberByKey("BUILD", IgorInfo(0)) < 36043)
#define -- error please update to a later nightly build
#endif

// https://www.wavemetrics.com/user/tony

// To prettify selected code: cmd-5 (Mac), ctrl-5 (Windows)

Constant kMinWordLengthDefault = 3

Menu "Edit"
    "Prettify Clipboard Code", /Q, PutScrapText Prettify#PrettifyCode(GetScrapText())
    "Prettify Selected Code/5", /Q, PrettifySelection()
End

Function PrettifySelection()
   
    String strScrap = GetScrapText()
    PutScrapText ""
    DoIgorMenu "Edit" "Copy"
    PutScrapText PrettifyCode(GetScrapText())
    DoIgorMenu "Edit" "Paste"
    PutScrapText strScrap // leave clipboard state unchanged
End

// Returns a string with known function and operation names case-corrected.
// Commented and quoted text is ignored. Trailing whitespace is eradicated.
// Blocks of code representing proc pictures are ignored, provided that the
// selection starts before the Picture keyword.
Function /S PrettifyCode(String strText, [string eol, string ignoreList, string customList, variable minWordLength])
   
    if(strlen(strText)==0)
        return ""
    endif
   
    if(ParamIsDefault(eol))
        eol = "\r"
    endif
   
    if(ParamIsDefault(ignoreList))
        ignoreList = ""
    endif

    if(ParamIsDefault(customList))
        customList = ""
    endif
   
    if(ParamIsDefault(minWordLength))
        minWordLength = kMinWordLengthDefault
    endif
   
    String strList = FunctionList("*",";","") + OperationList("*",";","")
    String keyWords="End;EndMacro;EndStructure;Function;Macro;Picture;Proc;Structure;Window;"
    // keyWords+="#if;#endif;#ifdef;#ifndef;#include;#pragma;#undef;" // can't match these as whole words
    keyWords+="Constant;DoPrompt;GalleryGlobal;hide;IgorVersion;IndependentModule;Menu;ModuleName;MultiThread;"
    keyWords+="Override;Popup;ProcGlobal;Prompt;root;rtGlobals;static;Strconstant;Submenu;TextEncoding;ThreadSafe;version;"
    String subtypes="ButtonControl;CameraWindow;CDFFunc;CheckboxControl;CursorStyle;DrawUserShape;FitFunc;"
    subtypes+="GizmoPlot;Graph;GraphMarquee;GraphStyle;GridStyle;Layout;LayoutMarquee;LayoutStyle;ListBoxControl;"
    subtypes+="Panel;PopupMenuControl;SetVariableControl;SliderControl;TabControl;Table;TableStyle;"
    String objectRefs="DFREF;FUNCREF;NVAR;STRUCT;SVAR;WAVE;"
    String flowControl="AbortOnRTE;AbortOnValue;break;case;catch;continue;default;do;else;elseif;endfor;endif;"
    flowControl+="endswitch;endtry;for;if;return;strswitch;switch;try;while;"
    strList+=keyWords+subtypes+objectRefs+flowControl

    // remove words in ignoreList, substitute or add words in customList
    strList = RemoveFromList(ignoreList, strList, ";", 0)
    strList = RemoveFromList(customList, strList, ";", 0) + customList
    WAVE /T wList = ListToTextWave(strList, ";")
    // remove words with fewer than kMinWordLength characters
    Variable i
    for(i=numpnts(wList)-1; i>=0; i-=1)
        if(strlen(wList[i]) < minWordLength)
            DeletePoints i, 1, wList
        endif
    endfor
   
    Variable endsWithReturn = (cmpstr(strText[strlen(strText)-1], eol) == 0)
    WAVE /T wText = ListToTextWave(strText, eol)
    Variable lastLine = numpnts(wText) - 1
    wText += SelectString(p<(lastLine) || endsWithReturn, "", eol)

    Make/FREE/N=(DimSize(wText, 0)) wIgnore

    CreateMask(wText, wIgnore, "Picture", "End")
    CreateMask(wText, wIgnore, "Window", "EndMacro")

    Multithread wText = PrettifyLine(wText, wList, wIgnore, eol)
    wfprintf strText, "%s", wText
    return strText
End

/// Creates a mask wave in wIgnore
///
/// wIgnore will have 1 if the line should be ignored, 0 otherwise
Function CreateMask(WAVE/T wText, WAVE wIgnore, string beginStr, string endStr)

    string beginRegExp = "(?i)^\s*\\Q" + beginStr + "\\E\b"
    string endRegExp = "(?i)^\s*\\Q" + endStr + "\\E\b"
    Variable vLine, i, numLines

    numLines = DimSize(wText, 0)

    // create a mask for lines that should not be altered
    Grep /Q/INDX/E=beginRegExp wText
    WAVE W_Index
   
    for (i=0;i<numpnts(W_Index);i+=1)
        vLine=w_Index[i]+1 // the line that starts the Picture binary as ASCII
        if(vLine>numLines)
            break
        endif
        do
            wIgnore[vLine]=1
            vLine+=1
        while(vLine<numLines && GrepString(wText[vLine],endRegExp)==0)
    endfor
End

// strLine should include \r if one was present in input text
threadsafe Function /S PrettifyLine(String strLine, WAVE /T wordList, Variable ignore, string eol)

    string nonWhitespace, whitespace

    if(ignore)
        return strLine
    endif
   
    WAVE /B mask = QuoteAndCommentMask(strLine)
    String strCode="", word=""
    int bytes = -1, len=numpnts(mask)
    // find position of last unmasked character in line
    Duplicate /free mask rmask
    Reverse mask /D=rmask
    FindValue /I=0 rmask // find first 0 in reversed mask wave
    if(v_value==-1) // entire line is masked
        strCode=""
    else
        strCode=strLine[0,len-1-v_value]
    endif
    // strCode should contain any code but no commented text

    do // loop through words in line
        SplitString /E="([[:alnum:]]+)(.*)" strCode, word, strCode
        FindValue /TEXT=word/TXOP=4/Z wordList
        if(V_value > -1)
            strLine = ReplaceWord(word, strLine, wordList[V_value], mask)
        endif
    while(strlen(strCode))

    SplitString/E=("^(?U)(.*)([[:space:]]*)$") strLine, nonWhitespace, whitespace
   
    return nonWhitespace + eol
End

ThreadSafe Function /WAVE QuoteAndCommentMask(String strLine)
       
    Variable startByte, endByte, len
    len=strlen(strLine)
    Make /B/free/N=(len) mask=0
    startByte=0
    do
        startByte=strsearch(strLine, "\"", startByte)
        if(startByte==-1) // no more quoted text before end of strLine
            break
        endif
        endByte=startByte
        do
            endByte=strsearch(strLine, "\"", endByte+1)
            if(endByte==-1)
                endByte=len-1
                break
            endif
            // endByte is possible end of quote
            // remove escaped backslashes!
            strLine[startByte,endByte]=ReplaceString("\\\\", strLine[startByte,endByte], "  ")
        while(cmpstr(strLine[endByte-1], "\\")==0) // ignore escaped quotes
        // found end of quote
        mask[startByte, endByte]=1
        startByte=endByte+1
    while(1) // look for next quoted text
    // quoted text is now masked
    startByte=0
    do
        startByte=strsearch(strLine, "//", startByte)
        if(startByte==-1) // no comment before end of strLine, we're done
            return mask
        endif
        if(mask[startByte]==1)
            startByte+=1
            continue
        endif
        // startByte is start of comment
        mask[startByte,len-1]=1
        return mask
    while(1)
End

// Wrapper for ReplaceString, replaces whole words
// characters inStr[i] where mask[i]==1 are ignored
ThreadSafe Function /S ReplaceWord(replaceThisWord, inStr, withThisWord, mask)
    String replaceThisWord, inStr, withThisWord
    WAVE /B mask
   
    String substring=""
    Variable startByte=0, endByte=0

    do // loop through matches
        startByte = strsearch(inStr, replaceThisWord, endByte, 2)
        if(startByte == -1)
            return inStr
        endif
        endByte = startByte + strlen(replaceThisWord)
        if(mask[startbyte]==1)
            continue
        endif
        substring = inStr[startByte-1, endByte+1] // will be clipped to 0, strlen(inStr)
        // check that match is whole word
        if(GrepString(substring, "(?i)\\b" + replaceThisWord + "\\b"))
            substring = ReplaceString(replaceThisWord, substring, withThisWord)
        endif
        inStr = inStr[0, startByte-2] + substring + inStr[endByte+2, Inf]
    while(1)
End

Function/S LoadFile(string fullPath)

    variable fnum
    string data

    Open/R/Z=1 fnum as fullPath

    FStatus fnum
    data = ""
    data = PadString(data, V_logEOF, 0x20)
    FBinRead fnum, data
    Close fnum
   
    return data
End

Function WriteFile(string fullPath, string contents)
   
    variable fnum
   
    Open/Z fnum as fullPath
    FBinWrite fnum, contents
    Close fnum
End

Function/S GetIgorStylePath(string path)

    return ParseFilepath(5, path, ":", 0, 0)
End

Function PrettifyFolder(string folder, [string eol, string customList, string ignoreList, variable minWordLength])

    string files, file, contents, contentsPretty
    variable numFiles, i
       
    if(ParamIsDefault(eol))
        eol = "\r"
    endif
   
    if(ParamIsDefault(ignoreList))
        ignoreList = ""
    endif

    if(ParamIsDefault(customList))
        customList = ""
    endif
   
    if(ParamIsDefault(minWordLength))
        minWordLength = kMinWordLengthDefault
    endif

    folder = GetIgorStylePath(folder)

    NewPath/O/Q tempFolder, folder
   
    files = IndexedFile(tempFolder, -1, ".ipf")
    numFiles = ItemsInList(files)
    for(i = 0; i < numFiles; i += 1)
        file = folder + ":" + StringFromList(i, files)
        contents = LoadFile(file)
        contentsPretty = Prettify#PrettifyCode(contents, eol = eol, customList = customList, ignoreList = ignoreList, minWordLength = minWordLength)
        WriteFile(file, contentsPretty)
    endfor
   
    KillPath/Z tempFolder
End

 

In reply to by thomas_braun

Very nice. I wouldn't have had the confidence to let it loose on a collection of files like that. Good to know that it seems to work.

Why do you mask window macros?

There is a missing semicolon following MultiThread in the version you forked (1.5).

For trailing whitespace, i would retain the conditional to allow whitespace on a line that doesn't have non-whitespace:

if(GrepString(strLine, "\S[[:blank:]]+\r$"))

My aim was to remove truly trailing whitespace, but keep indentation tabs in otherwise empty lines. Maybe you can do this with some tricksy regex?

I don't use GitHub for Igor code.

> Why do you mask window macros?

I don't want to prettify machine generated code. As I also don't edit window macros manually but regenerate them.


> There is a missing semicolon following MultiThread in the version you forked (1.5).

Thanks, fixed.

> For trailing whitespace, i would retain the conditional to allow whitespace on a line that doesn't have non-whitespace.

I'm using git's builtin whitespace checks and these also flag trailing whitespace on otherwise empty lines. What is the purpose of keeping that?

If you want to keep that whitespace I would add an early return like

if(GrepString(strLine, "^[[:space:]]*$"))
  return strLine
endif

 

In reply to by thomas_braun

Igor's editor does auto-indentation by inserting tabs on a new line to match the indentation level of the preceding line. I prefer not to erase that. It could be a configurable option, maybe retaining tabs but removing trailing spaces.

In reply to by thomas_braun

I thought I would incorporate the extensions to the code provided by @thomas_braun into the TextEditTools procedure, but came across a couple of problems.

Most importantly, I think that procedure files that have been edited in Igor could have either line feed or returns (ASCII 10 or 13) at the end of lines. The wrong value for eol will truncate the file contents. So don't use the PrettifyFolder function unless you are absolutely sure that every ipf file has the same kind of line break! The code overwrites all the ipf files in the folder, so unless you know what you're doing it's best to avoid using that function entirely.

I think it would be safer to add a robust check for the type of line break for each file.

I checked that byte order mark is left intact.

The revised method for stripping trailing whitespace doesn't allow for selecting and prettifying just a part of a line, but that is easily fixed.

Thanks for looking into that again.

First of all, I always have all code in version control. So even if a random tool destroys my code I don't loose any work. Maybe I should have pointed that our more clearly.

And I also standardize EOLs using the stock git support.

If you want to handle files with differing EOLs I think the best approach is to first normalize them, https://github.com/AllenInstitute/MIES/blob/dacf218d93523ee2c8a7acaafea… should do the trick.

In reply to by thomas_braun

Yeah, I figured that you would be coding in a proper development setup, but many Igor users are likely to be less aware of such things.

I was surprised to discover that my collection of ipf files has variable EOLs. Occasionally I code in Sublime Text, but I suspect that one cannot rely even on files that were created by Igor on the same platform to always have the same EOL, so users should be cautious!

For anyone following this thread, none of this affects the functionality of the prettify function used within Igor, i.e. selecting text within a procedure window and prettifying selected code is unaffected by and has no effect on the structure of the saved file.

Forum

Support

Gallery

Igor Pro 8

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More