Advice on breaking bad programming habits

As a starting point, consider moving the procedure file in a representative experiment to a stand-alone procedure.

• Open an experiment with the most recent or most reliable procedure file.
• Open the main procedure window.
• Choose the menu option File->Save Procedure Copy.
• Save the copy of the procedure in a folder in the User Procedures folder of the WaveMetrics folder. Title the folder with a respectable, short name to represent the theme of the entire procedure. For example, Analysis Tools or Curve Fitting Process or ... Title the procedure with a respectable, short name to represent it, for example also Main Analysis Procedure or equivalent.
• Start a new Igor Pro experiment.
• Open the main procedure window and put the #include "Main Analysis Procedure" statement in the beginning portion of the window.
• Recompile the code.

You should now have the equivalent of access to the procedures without having to duplicate the experiment containing the main procedure.

The next step begins the process for development. At the top portion of the Main Analysis Procedure, include the pragma #version=1.00. Save the experiment that you just opened with a name equivalent to below.

-> Develop Main Analysis v1.00.pxp

You now have an experiment where you devote your efforts solely to developing and improving the procedures for analysis, not to muck about in the code while analyzing data. The work you do on the #include procedure in the Develop experiment transfers to all future experiments that have the #include "Main Analysis Procedure" pragma.

At this point, each time before you open the Develop experiment, decide what target you will resolve. Targets can include such things as ->add a feature, ->fix a bug, ->streamline a certain code segment, or ->remove clutter in something. Also, every time before you open the Develop ... experiment, create a ZIP archive of the main procedure file as a backup. When you reach a desired milestone in development or improvement, increment the version number in the procedure file and save the experiment with the incremented version number. Approaches vary on setting incremental version numbers. I typically increment by steps of 0.10 for bug fixes or new features, and I increment all other work by steps of 0.01.

To summarize, the important points to prevent being overwhelmed are a) extract the cores procedures to a separate procedure file that can be called by #include, b) develop a set of milestones that you attack one at a time, c) hold to a robust approach to capture snapshot backups of the main procedure file before you undertake any revisions on it, and d) maintain a systematic approach to versioning the main procedure file (and development experiment) to mark your success after reaching a milestone.

I hope my suggestions might help you get started on a more structured approach to your coding efforts.

I can give tons of good advice ;)

But I'm also wondering what you think is going wrong with your current approach.

Jeffrey,  thats a good start - many thanks.

Thomas,

the problem for me is one of managing my programming developments which have become unwieldy: for example in 2024 with developments over 50 experiments totally over 13 GB.  Present version is 0.5GB with 20 procedures and hundreds of functions, some used and some defunct.  No control on function versions and modifcations ( only commented changes) .  I am the sole user. 

Keeping tabs on updates to functions is probably the biggest issue.

"...maintain a systematic approach to versioning the main procedure file (and development experiment) to mark your success after reaching a milestone."

Yes, as Jeffrey pointed out versioning is particularly important in this kind of projects.  I remember when I started developing in Igor one of my first projects dealed with processing data files issued by a custom acquisition program originally written by someone else.  As the experiment evolved new informations were added to the files and I was updating the processing routines accordingly. However initially there was no version info in the custom files so that at some point the latest version of my processing software was no longer compatible with older files which posed many problems. Since then when I develop my own aquisition programs  the file format/acquisition software versions are coded within the first lines (or bytes) of the file, my processing software starts reading those bytes and selects the appropriate version of the loader/processing routines.   Now I can semlessly load and process files stored 15 years ago, even when I don't remember which file format version was used at the time.

> Keeping tabs on updates to functions is probably the biggest issue.

As a follow up, you might also want to consider whether you can improve on how you track your modifications. One method is to keep the records of changes in the procedure files themselves. For example, you could include a section near the start of a procedure file that keeps track of every modification that you make on it. Another method is to keep versioning records in a separate document system. Whichever approach you take, be systematic. Note the changes promptly when they are made, categorize the general type of change made (e.g. bug fix, code improvement, UI improvement, new feature, code pruning, ...) in addition to putting at least a brief note about the specific change(s), and date stamp the change record.

A very minor addition to the suggestions given so far is to take advantage of the Code Marker Comments tool in the built in code editor.   It can be accessed by the sort of map marker icon in the upper left of the editor just to the right of the back and forward arrows..

This can be used to jump around to significant locations in your code.  I use it to mark the end of my header comments and the locations of other things such as structure, menu, and constant definitions.

see: 

Thanks All,  just the type of advice i hoped for!

I find that the best motivation for writing cleaner code is to share code with others.

My advice would be to break everything down into as-compact-as-possible procedure files and store these in User Procedures. This really echoes the advice of others, and the wording of your question suggests that that is where you are heading anyway. If, as the sole user, you work always on the same computer, I cannot see any drawback to doing things this way. If you currently move your self-contained pxp files between computers, you will need to think about how to keep the User Procedures folders synchronized, but it's not a very formidable task.

A system that I find helpful is to keep a 'default includes' procedure file in the Igor Procedures folder. This ensures that my personal selection of go-to packages is always loaded. I also use that file to add a handful of useful one-liner menu items that I like to have available, like this:

#pragma version=1.02
 
// Procedure files to load at startup.
// #include "file name" for user procedures,  
// #include <file name> for WaveMetrics procedure files
 
// *** programming & procedure file maintenance ***
#include "UserProcLoader"
 
// and so on, a list of file loaders and other utility functions to be included
 
// *** menu definitions ***
menu "Misc", hideable
    "Prepare project release...", /Q, updater#PrepareProjectRelease()
    "Disable Threadsafe Support", Execute/P/Q "SetIgorOption poundDefine=THREADING_DISABLED"; Execute/P/Q "SetIgorOption DisableThreadsafe=1"
    "Enable Threadsafe Support",  Execute/P/Q "SetIgorOption poundUnDefine=THREADING_DISABLED"; Execute/P/Q "SetIgorOption DisableThreadsafe=0"
end

I strongly recommend sticking with the default User Procedures folder for storing all other procedures.

A system that I find helpful is to keep a 'default includes' procedure file in the Igor Procedures folder. 

That's a good idea, thanks Tony.

I strongly recommend sticking with the default User Procedures folder for storing all other procedures.

...Alternatively, I just put a shortcut in User Procedures that points to my procedures. Those are saved in a separate, automatically backed up folder.

One other addition to the advice above: when coding for scientific work, it is often especially important to keep track of what version was used for what calculation / at what time. For this, I backup my code folder at major milestones, and also version-number with the calendar date to avoid confusion (e.g. this post is version 20251028).

Just to piggyback on the excellent advice from everyone. Version control, version control, version control. And something that others haven't brought up is writing unit tests. If you write tests for each non-static function that can execute automatically and call the function without calling any other non-static function, it will force you to write cleaner code. The combo of version control and easy to run tests is the key to excellent process.

-Kris

A bit late coming back to this but has anyone written a function to do the menu option File->Save Procedure Copy on all user procedures?

I'm not sure if, by "user procedures", you mean procedures in the "User Procedures" folder or any non-Igor procecure file.

Assuming the former, here is an approach:

// *** Note *** If your "User Procedure Files" folder contains aliases or shortcuts,
// this copies the aliases or shortcuts, not the original files
Function CopyUserProceduresFolder()
    String sourcePath = SpecialDirPath("Igor Pro User Files", 0, 0, 0)
    sourcePath += "User Procedures"
 
    String destPath = SpecialDirPath("Documents", 0, 0, 0)
    destPath += "Igor Pro User Files Copy"
    destPath += " " + Secs2Date(DateTime,-2)    // YYYY-MM-DD
    
    // Overwrites any pre-existing copy
    CopyFolder/O sourcePath as destPath
End

I'm not sure if, by "user procedures", you mean procedures in the "User Procedures" folder or any non-Igor procecure file.

 Didn't explain that too well.

From an open pxp experiment rather than using the File>Save Procedure Copy... key entry on all of my "user written" procedures, I was looking for a function that would take each of my procedures and save to a subfolder in the designated Igor Pro User files folder.

This function creates a list of all of my  procedures in (26 off) for a basis ....

Function ListUserProcedures()
    // MTG 18 July 2015 v2
    // 
    variable i,j, lenProcItem, match    
    String FullProcList = WinList("*",";","WIN:128")    // list of all procedure windows  
    string UserProcList =""                             // List of user procedures 
    Notebook UserFunctionMacroLists text = "#User Procedures\r"
    variable numItems = ItemsInList(FullProcList)
    string procItem 
 
    // remove .ipf files from list
    for(i=0;i<numItems;i+=1)        
        procItem = stringFromList(i, FullProcList)
        lenprocItem = strlen(procItem)
        match = cmpstr(procItem[lenProcItem-4], ".")
        if (match > 0)
            UserProcList +=ProcItem+"\r"
        endif
    endfor
 
    Notebook UserFunctionMacroLists text = UserProcList+"\r\r"
End 

@Mike

@Mike

Mike German wrote:

I'm not sure if, by "user procedures", you mean procedures in the "User Procedures" folder or any non-Igor procecure file.

 Didn't explain that too well.

Mike, why not just Adopt all the procedures (CTRL+SHIFT+File menu) into the file. Then you don't need to keep track of external files, and can be sure what version was used. 

Alternatively, if you are using #include for your files, then you can automatically/manually backup the included folder periodically. 

Mike German wrote:

This function creates a list of all of my  procedures in (26 off) for a basis ....

That doesn't look like a very robust way to identify user procedures.

Do you want to make copies of all procedure files in the current experiment that are not wavemetrics procedures and not the procedure window (this could include files stored anywhere other than Igor Procedures), or perhaps all open procedure files that are stored in User Procedures folder? Do you need to retain folder structure?

I would check file path against Igor Procedures (for first option), or else against User Procedures, to refine the list.

get file paths using GetWindow/Z $strWin file

identify files of interest, probably using SpecialDirPath("Igor Pro User Files",0,0,0)

Use NewPath if you want user to select a destination folder, then use CopyFile for each procedure in the refined list.

You might want to check for unsaved changes, using GetWindow $strWin needupdate

If you want to back up User Procedures folder, that's probably most easily done outside of Igor.

If you're interested in synchronising folders from within Igor, there's a code snippet here. It's not designed for copying files into subfolders, though.

This comment from your "ListUserProcedures" function:

makes me think that you want to save a copy of only packed procedure files (stored in the experiment file). By excluding procedure windows whose titles include ".", you are excluding WaveMetrics procedure files (e.g., "HDF5 Utilities.ipf") and also your own standalone procedure files.

Is my understanding of your intent correct?

The reason for wanting the 26 procedures into one folder on disk is that, because of my bad habits I have 3 experiments which are nominally identical other than mods to different procedures.  If I had just these procedures containing code saved to separate folders on a PC I can use WinMerge to compare and identify differences.

As this is a one off (hopefully) I guess I bite the bullet and do it manually.  I was just interested in how it could be done.

Thanks for all advice  

Attached is a procedure for comparing different ipfs, written by James Allan in the pre-git era. Perhaps this is useful for identifying differences.

tony wrote:

Attached is a procedure for comparing different ipfs, written by James Allan in the pre-git era. Perhaps this is useful for identifying differences.

Yes that would be helpful when I have extracted to windows folders all of the 26 procedures from  3 or 4 experiments.

I've just come across DoIgorMenu which "allows an Igor programmer to invoke Igor's built-in menu items. This is useful for bringing up Igor's built-in dialogs under program control"  (I found it in Help file: "C:Program Files:WaveMetrics:Igor Pro 10 Folder:Igor Help Files:Igor Reference.ihf" (Topic: DoIgorMenu)

 I tried  DolgorMenu "File", "Save procedure Copy"  from the command line but this doesn't work.  The hope would be to use this in a macro to save all procedures to a named folder.

 I tried  DolgorMenu "File", "Save procedure Copy"  from the command line but this doesn't work.  The hope would be to use this in a macro to save all procedures to a named folder.

In place of trying to use 'Save Procedure Copy' programmatically, you can hack your way around this with the sometimes-very-handy ProcedureText() function, which extracts the text of the procedure file, and then you can just write that text to a new file:

function saveProcedure(String procedureName, String savePath)
    String text = ProcedureText("", 0, procedureName)
    Variable ref
    Open ref as savePath  // savePath is: "path/to/new/proc.ipf"
    FBinWrite ref, text
    Close ref
end