Prettify code

Case-corrects Igor function and operation names, and removes trailing whitespace.

Note that the file containing this code will be loaded as an independent module. You'll have to show hidden items in the procedure browser to see the code.

Edit: I've removed the code snippet from this post. Follow the link to the 'text editing tools' code snippet to find a more recent version.

 

The updated version posted above ignores quoted text and blocks of code representing proc pictures, provided that the selection starts before the Picture keyword. If I have got this right, Prettify will case-correct only words that are colored by Igor's syntax highlighting as operation or function names.

July 10, 2020 at 01:47 am - Permalink

Great that you have everything in one package now. Feel free to add my align comments code as well, so that we have everything in one neat file.

July 12, 2020 at 09:16 pm - Permalink

I posted the file with everything as a new thread in code snippets forum. Any user can edit.

July 13, 2020 at 03:47 am - Permalink

I've played around with it. And it is quite useful! Are you using github for other projects? Collaboration is more easily there IMHO.

I'm posting below a modified version.

Changes:

- Properly bailout on old IP

- Factor out mask creation and ignore window recreation macros as well

- Made hardcoded customization constants into optional parameters

- Added code to prettify all procedure files in a disc folder

- Handle ignoreList without trailing semicolon

- Simplify trailing whitespace removal

- Allow custom eol character

With that I could prettify some 60k LOC project using

Prettify#prettifyfolder("e:projekte:mies-igor:Packages:MIES", eol = "\n", customList = "variable;string;static;threadsafe;StringMatch", ignoreList="graph;proc;panel")

It does still compile and I could not yet spot any errors.
 

#pragma TextEncoding="UTF-8"
#pragma rtGlobals=3
#pragma IgorVersion=8
#pragma IndependentModule=Prettify
#pragma version=1.5
 
#if (NumberByKey("BUILD", IgorInfo(0)) < 36043)
#define -- error please update to a later nightly build
#endif
 
// https://www.wavemetrics.com/user/tony
 
// To prettify selected code: cmd-5 (Mac), ctrl-5 (Windows)
 
Constant kMinWordLengthDefault = 3
 
Menu "Edit"
    "Prettify Clipboard Code", /Q, PutScrapText Prettify#PrettifyCode(GetScrapText())
    "Prettify Selected Code/5", /Q, PrettifySelection()
End
 
Function PrettifySelection()
   
    String strScrap = GetScrapText()
    PutScrapText ""
    DoIgorMenu "Edit" "Copy"
    PutScrapText PrettifyCode(GetScrapText())
    DoIgorMenu "Edit" "Paste"
    PutScrapText strScrap // leave clipboard state unchanged
End
 
// Returns a string with known function and operation names case-corrected.
// Commented and quoted text is ignored. Trailing whitespace is eradicated.
// Blocks of code representing proc pictures are ignored, provided that the
// selection starts before the Picture keyword.
Function /S PrettifyCode(String strText, [string eol, string ignoreList, string customList, variable minWordLength])
   
    if(strlen(strText)==0)
        return ""
    endif
    
    if(ParamIsDefault(eol))
        eol = "\r"
    endif
    
    if(ParamIsDefault(ignoreList))
        ignoreList = ""
    endif
 
    if(ParamIsDefault(customList))
        customList = ""
    endif
    
    if(ParamIsDefault(minWordLength))
        minWordLength = kMinWordLengthDefault
    endif
   
    String strList = FunctionList("*",";","") + OperationList("*",";","")
    String keyWords="End;EndMacro;EndStructure;Function;Macro;Picture;Proc;Structure;Window;"
    // keyWords+="#if;#endif;#ifdef;#ifndef;#include;#pragma;#undef;" // can't match these as whole words
    keyWords+="Constant;DoPrompt;GalleryGlobal;hide;IgorVersion;IndependentModule;Menu;ModuleName;MultiThread;"
    keyWords+="Override;Popup;ProcGlobal;Prompt;root;rtGlobals;static;Strconstant;Submenu;TextEncoding;ThreadSafe;version;"
    String subtypes="ButtonControl;CameraWindow;CDFFunc;CheckboxControl;CursorStyle;DrawUserShape;FitFunc;"
    subtypes+="GizmoPlot;Graph;GraphMarquee;GraphStyle;GridStyle;Layout;LayoutMarquee;LayoutStyle;ListBoxControl;"
    subtypes+="Panel;PopupMenuControl;SetVariableControl;SliderControl;TabControl;Table;TableStyle;"
    String objectRefs="DFREF;FUNCREF;NVAR;STRUCT;SVAR;WAVE;"
    String flowControl="AbortOnRTE;AbortOnValue;break;case;catch;continue;default;do;else;elseif;endfor;endif;"
    flowControl+="endswitch;endtry;for;if;return;strswitch;switch;try;while;"
    strList+=keyWords+subtypes+objectRefs+flowControl
 
    // remove words in ignoreList, substitute or add words in customList
    strList = RemoveFromList(ignoreList, strList, ";", 0)
    strList = RemoveFromList(customList, strList, ";", 0) + customList
    WAVE /T wList = ListToTextWave(strList, ";")
    // remove words with fewer than kMinWordLength characters
    Variable i
    for(i=numpnts(wList)-1; i>=0; i-=1)
        if(strlen(wList[i]) < minWordLength)
            DeletePoints i, 1, wList
        endif
    endfor
   
    Variable endsWithReturn = (cmpstr(strText[strlen(strText)-1], eol) == 0)
    WAVE /T wText = ListToTextWave(strText, eol)
    Variable lastLine = numpnts(wText) - 1
    wText += SelectString(p<(lastLine) || endsWithReturn, "", eol)
 
    Make/FREE/N=(DimSize(wText, 0)) wIgnore
 
    CreateMask(wText, wIgnore, "Picture", "End")
    CreateMask(wText, wIgnore, "Window", "EndMacro")
 
    Multithread wText = PrettifyLine(wText, wList, wIgnore, eol)
    wfprintf strText, "%s", wText
    return strText
End
 
/// Creates a mask wave in wIgnore
///
/// wIgnore will have 1 if the line should be ignored, 0 otherwise
Function CreateMask(WAVE/T wText, WAVE wIgnore, string beginStr, string endStr)
 
    string beginRegExp = "(?i)^\s*\\Q" + beginStr + "\\E\b"
    string endRegExp = "(?i)^\s*\\Q" + endStr + "\\E\b"
    Variable vLine, i, numLines
 
    numLines = DimSize(wText, 0)
 
    // create a mask for lines that should not be altered
    Grep /Q/INDX/E=beginRegExp wText
    WAVE W_Index
   
    for (i=0;i<numpnts(W_Index);i+=1)
        vLine=w_Index[i]+1 // the line that starts the Picture binary as ASCII
        if(vLine>numLines)
            break
        endif
        do
            wIgnore[vLine]=1
            vLine+=1
        while(vLine<numLines && GrepString(wText[vLine],endRegExp)==0)
    endfor
End
 
// strLine should include \r if one was present in input text
threadsafe Function /S PrettifyLine(String strLine, WAVE /T wordList, Variable ignore, string eol)
 
    string nonWhitespace, whitespace
 
    if(ignore)
        return strLine
    endif
   
    WAVE /B mask = QuoteAndCommentMask(strLine)
    String strCode="", word=""
    int bytes = -1, len=numpnts(mask)
    // find position of last unmasked character in line
    Duplicate /free mask rmask
    Reverse mask /D=rmask
    FindValue /I=0 rmask // find first 0 in reversed mask wave
    if(v_value==-1) // entire line is masked
        strCode=""
    else
        strCode=strLine[0,len-1-v_value]
    endif
    // strCode should contain any code but no commented text
 
    do // loop through words in line
        SplitString /E="([[:alnum:]]+)(.*)" strCode, word, strCode
        FindValue /TEXT=word/TXOP=4/Z wordList
        if(V_value > -1)
            strLine = ReplaceWord(word, strLine, wordList[V_value], mask)
        endif
    while(strlen(strCode))
 
    SplitString/E=("^(?U)(.*)([[:space:]]*)$") strLine, nonWhitespace, whitespace
    
    return nonWhitespace + eol
End
 
ThreadSafe Function /WAVE QuoteAndCommentMask(String strLine)
       
    Variable startByte, endByte, len
    len=strlen(strLine)
    Make /B/free/N=(len) mask=0
    startByte=0
    do
        startByte=strsearch(strLine, "\"", startByte)
        if(startByte==-1) // no more quoted text before end of strLine
            break
        endif
        endByte=startByte
        do
            endByte=strsearch(strLine, "\"", endByte+1)
            if(endByte==-1)
                endByte=len-1
                break
            endif
            // endByte is possible end of quote
            // remove escaped backslashes!
            strLine[startByte,endByte]=ReplaceString("\\\\", strLine[startByte,endByte], "  ")
        while(cmpstr(strLine[endByte-1], "\\")==0) // ignore escaped quotes
        // found end of quote
        mask[startByte, endByte]=1
        startByte=endByte+1
    while(1) // look for next quoted text
    // quoted text is now masked
    startByte=0
    do
        startByte=strsearch(strLine, "//", startByte)
        if(startByte==-1) // no comment before end of strLine, we're done
            return mask
        endif
        if(mask[startByte]==1)
            startByte+=1
            continue
        endif
        // startByte is start of comment
        mask[startByte,len-1]=1
        return mask
    while(1)
End
 
// Wrapper for ReplaceString, replaces whole words
// characters inStr[i] where mask[i]==1 are ignored
ThreadSafe Function /S ReplaceWord(replaceThisWord, inStr, withThisWord, mask)
    String replaceThisWord, inStr, withThisWord
    WAVE /B mask
   
    String substring=""
    Variable startByte=0, endByte=0
 
    do // loop through matches
        startByte = strsearch(inStr, replaceThisWord, endByte, 2)
        if(startByte == -1)
            return inStr
        endif
        endByte = startByte + strlen(replaceThisWord)
        if(mask[startbyte]==1)
            continue
        endif
        substring = inStr[startByte-1, endByte+1] // will be clipped to 0, strlen(inStr)
        // check that match is whole word
        if(GrepString(substring, "(?i)\\b" + replaceThisWord + "\\b"))
            substring = ReplaceString(replaceThisWord, substring, withThisWord)
        endif
        inStr = inStr[0, startByte-2] + substring + inStr[endByte+2, Inf]
    while(1)
End
 
Function/S LoadFile(string fullPath)
 
    variable fnum
    string data
 
    Open/R/Z=1 fnum as fullPath
 
    FStatus fnum
    data = ""
    data = PadString(data, V_logEOF, 0x20)
    FBinRead fnum, data
    Close fnum
    
    return data
End
 
Function WriteFile(string fullPath, string contents)
    
    variable fnum
    
    Open/Z fnum as fullPath
    FBinWrite fnum, contents
    Close fnum
End
 
Function/S GetIgorStylePath(string path)
 
    return ParseFilepath(5, path, ":", 0, 0)
End
 
Function PrettifyFolder(string folder, [string eol, string customList, string ignoreList, variable minWordLength])
 
    string files, file, contents, contentsPretty
    variable numFiles, i
        
    if(ParamIsDefault(eol))
        eol = "\r"
    endif
    
    if(ParamIsDefault(ignoreList))
        ignoreList = ""
    endif
 
    if(ParamIsDefault(customList))
        customList = ""
    endif
    
    if(ParamIsDefault(minWordLength))
        minWordLength = kMinWordLengthDefault
    endif
 
    folder = GetIgorStylePath(folder)
 
    NewPath/O/Q tempFolder, folder
    
    files = IndexedFile(tempFolder, -1, ".ipf")
    numFiles = ItemsInList(files)
    for(i = 0; i < numFiles; i += 1)
        file = folder + ":" + StringFromList(i, files)
        contents = LoadFile(file)
        contentsPretty = Prettify#PrettifyCode(contents, eol = eol, customList = customList, ignoreList = ignoreList, minWordLength = minWordLength)
        WriteFile(file, contentsPretty)
    endfor
    
    KillPath/Z tempFolder
End

 

July 16, 2020 at 06:12 am - Permalink

Very nice. I wouldn't have had the confidence to let it loose on a collection of files like that. Good to know that it seems to work.

Why do you mask window macros?

There is a missing semicolon following MultiThread in the version you forked (1.5).

For trailing whitespace, i would retain the conditional to allow whitespace on a line that doesn't have non-whitespace:

if(GrepString(strLine, "\S[[:blank:]]+\r$"))

My aim was to remove truly trailing whitespace, but keep indentation tabs in otherwise empty lines. Maybe you can do this with some tricksy regex?

I don't use GitHub for Igor code.

July 16, 2020 at 10:19 am - Permalink

> Why do you mask window macros?

I don't want to prettify machine generated code. As I also don't edit window macros manually but regenerate them.


> There is a missing semicolon following MultiThread in the version you forked (1.5).

Thanks, fixed.

> For trailing whitespace, i would retain the conditional to allow whitespace on a line that doesn't have non-whitespace.

I'm using git's builtin whitespace checks and these also flag trailing whitespace on otherwise empty lines. What is the purpose of keeping that?

If you want to keep that whitespace I would add an early return like

if(GrepString(strLine, "^[[:space:]]*$"))
  return strLine
endif

 

July 16, 2020 at 10:47 am - Permalink

Igor's editor does auto-indentation by inserting tabs on a new line to match the indentation level of the preceding line. I prefer not to erase that. It could be a configurable option, maybe retaining tabs but removing trailing spaces.

July 16, 2020 at 11:54 pm - Permalink

I thought I would incorporate the extensions to the code provided by @thomas_braun into the TextEditTools procedure, but came across a couple of problems.

Most importantly, I think that procedure files that have been edited in Igor could have either line feed or returns (ASCII 10 or 13) at the end of lines. The wrong value for eol will truncate the file contents. So don't use the PrettifyFolder function unless you are absolutely sure that every ipf file has the same kind of line break! The code overwrites all the ipf files in the folder, so unless you know what you're doing it's best to avoid using that function entirely.

I think it would be safer to add a robust check for the type of line break for each file.

I checked that byte order mark is left intact.

The revised method for stripping trailing whitespace doesn't allow for selecting and prettifying just a part of a line, but that is easily fixed.

September 10, 2020 at 10:10 am - Permalink

Thanks for looking into that again.

First of all, I always have all code in version control. So even if a random tool destroys my code I don't loose any work. Maybe I should have pointed that our more clearly.

And I also standardize EOLs using the stock git support.

If you want to handle files with differing EOLs I think the best approach is to first normalize them, https://github.com/AllenInstitute/MIES/blob/dacf218d93523ee2c8a7acaafea… should do the trick.

September 10, 2020 at 03:18 pm - Permalink

Yeah, I figured that you would be coding in a proper development setup, but many Igor users are likely to be less aware of such things.

I was surprised to discover that my collection of ipf files has variable EOLs. Occasionally I code in Sublime Text, but I suspect that one cannot rely even on files that were created by Igor on the same platform to always have the same EOL, so users should be cautious!

For anyone following this thread, none of this affects the functionality of the prettify function used within Igor, i.e. selecting text within a procedure window and prettifying selected code is unaffected by and has no effect on the structure of the saved file.

September 11, 2020 at 02:34 am - Permalink

Forum

Support

Gallery

Igor Pro 9

Learn More

Igor XOP Toolkit

Learn More

Igor NIDAQ Tools MX

Learn More