Sending data via sockit

I tried posting this on the igor mailing list but it seems to be running slow....
I am trying to use sockit to send and receive data from a lightcrafter device. This is a micromirror controller. The documentation is poor and my knowledge of bits and bytes is sketchy. Nevertheless......

Apparently, using Matlab, the following should do something interesting....

Generating a TCPIP object:

tcpObject = tcpip(192.168.1.100,Port Number) %concerning to the manual the IP Address is fixed

fopen(tcoObject);

to send a command to the LC use following code:

fwrite(tcpObject,uint8(hex2dec(['02';'01';'01';'00';'01';'00';'00';'05']))); %switch display mode to static

fwrite(tcpObject,uint8(hex2dec(['02';'01';'01';'00';'01';'00';'01';'06']))); %siwtch display mode to internal

fwrite(tcpObject,uint8(hex2dec(['02';'01';'03';'00';'01';'00';'00';'07']))); %switch to internal pattern: checkboard


I can open an connection with sockit but I don't know how to replicate sending these hexadecimal numbers.
My attempt was simply to do the following:
•SKTSendSockitMsg("02 01 01 00 01 00 00 05", 2352)

in the history, I get
SOCKITmsg: wrote to socket 2352
02 01 01 00 01 00 00 05


Hopefully the problem is with my interpretation of the code rather than the connection.
Any suggestions?
Thanks,
Nick Hartell
There are some elementary mistakes you're making :-).
string msg="02 01 01 00 01 00 00 05"
print strlen(msg)
  23

You're actually sending 23 bytes, not the 8 you are trying to. What you need to send is the following:

string msg = num2char(0x02) + num2char(0x01) + num2char(0x01) + num2char(0x00) + num2char(0x01) + num2char(0x00) + num2char(0x00) + num2char(0x05)
print strlen(msg)
8


As a quicker way of creating the message you can do:
make/B/U/o msgwav = {0x02, 0x01, 0x01, 0x00, 0x01, 0x00, 0x00, 0x05}
string msg
//transfers binary data from msgwav to the msg string
sockitwavetostring msgwav, msg
print strlen(msg)
8


We can inspect the message you tried to send:
//look up wavetype for the first argument to the operation
•sockitstringtowave 2^3 + 2^6, "02 01 01 00 01 00 00 05"
edit W_stringtowave

Lookup the values in W_stringtowave and compare them with www.asciitable.com

Let me know if you need more help.
Thanks Andy,
That is very helpful.
However, it is still not working. I have done what you suggested but for some reason, I appear to be sending 9 bytes not 8.
If I send the following:

make/B/U/o msgwav3 = {0x02, 0x01, 0x03, 0x00, 0x01, 0x00, 0x00, 0x07}
sockitwavetostring msgwav3, msg3
SKTSendSockitMsg(msg3,SockitID)


I look using wireshark and it tells me that i have actually a data size of 9 bytes:
02 01 03 00 01 00 00 07 0a

I wonder where this last byte is coming from? Am I perhaps opening the sockit incorrectly and sending a terminating mark or something?
Any ideas?
Thanks,
Nick
0x0A is line feed.
Usually something like 0x0A 0x0D (LFCR) or 0x0D 0x0A (CRLF) is expected...
(btw: CR 0x0D aka 0d13 is usually the trigger for "something new" and is truncated. Hence "0x0A 0x0D" might appear only as "0x0A")


what does print strlen(msg3) say?
HJ
Thanks for the replies.
print strlen(msg3) returns 8.
One other thing to add. The protocol I am using is apparently RNDIS over USB which I think emulates TCP/IP. Could this be expecting something that sockit does not provide?

B
Nick
OK I have to fess up. My function for sending a message via sockit added a new line as this was needed for my other use of sockits with arduino's. oops.

So I now seem to be sending the correct data. Sadly, it still doesn't seem to respond.
Thanks for the help. I will report back if I have any success.
B
Nick
Let me know if you need (minor) SOCKIT modifications made. I would try communicating with the hardware via telnet first. If you can get telnet to work, then SOCKIT normally works.

A.
Right...... Another evening wasted trying to get this thing working to no avail.

I am fairly sure that I am sending the correct data via a packet because wireshark tells me the data payload is correct. However, I do not get an appropriate response back for commands that should cause the lightcrafter to send a response. I wonder then whether the method of sending the data packet is missing something or using a different basic protocol. I have tried other software to send packets and none of them work even though the packets I send are the same as those that I can see being sent on wireshark.

I have c code that describes the packeting procedure and apparently it works. I can post it here although i would need to include all the copyright stuff.
Andy, would you be willing to have a quick look at it as it might help to explain why commands sent through sockit are not working? I could send it to you offline if you prefer?
Thanks,
Best,
Nick
Sure, send if offline if it's small. Can't guarantee when I'll get to it. Sockit only works with TCP, not UDP.
I don't have matlab to test this out. I am going to try and compile the c code and see if that works. In the meantime, I will send you the c code. The relevant bit is quite small.
B
Nick

In reply to by andyfaff

I think I might have found a workaround

The idea is to sent a unix command from Igor executing a netcat routine to dump the data into a text file, then use Igor to pick it up from there. 
 

unixCmd = "nc -l -u 4450 &> '/Users/srot/desktop/test4453.txt'"//  ls '/Users/srot/desktop.test4450.txt'"

 
    String igorCmd
    sprintf igorCmd, "do shell script \"%s\"", unixCmd
    Print igorCmd       // For debugging only
    ExecuteScriptText/b/UNQ/W=1000/Z igorCmd

Unfortunately, I run into a problem that when I execute this. The netcat needs to be terminated manually by either pressing CRTL + c or by sending the command 

killall nc

Unfortunately Igor is stuck in the "listening mode" and cannot send that kill command. 

Any idea how to force Igor to abort that command and continue in the script?

 

Cheers

Silvan

I have edited your last post to put the commands in an Igor Code Snippet. It's good for more than just code.

You should be able to kill the process started by ExecuteScriptText by pressing Ctrl-Break or Shift-Esc (which I added when I got a keyboard that didn't have the Break key). Break may be labelled "Pause".

Your /W flag establishes a pretty long wait time. You might want to shorten it to something less than almost 17 minutes.

yes, but how can I avoid pressing Ctrl-Break or Shift-Esc, since I want this to run continuous, picking up the value in the txt file every second?

It seems ExecuteScriptText waits for a very long time (1000 s) here. I don't know the details of your approach, but can't you just set this to a very small time (<1 s) and let Igor break out immediately? If you are waiting for some kind of response, then this should be separated into another function, i.e., have a separate sender and listener. Are you using a background task within Igor for the communication? => If not, then that's the first thing you should look into:

DisplayHelpTopic "Background Tasks"

 

Unfortunately it still doesn't work this way. I created two functions DemoUnixShellCommand() and DemoUnixShellCommand2(), one to start the listening process on the port 4450, and the other one to terminate it. However, when I execute the DemoUnixShellCommand() even as a background task, Igor get's stuck in the listening mode and does not execute any further background tasks to abort the listening mode. Any idea how to change that?

 

 

 

Function/S DemoUnixShellCommand()
    String unixCmd
//  unixCmd = "ls '/Applications/Igor Pro 8 Folder'"
    unixCmd = "nc -l -u 4450 &> '/Users/srot/desktop/test4453.txt'"//  ls '/Users/srot/desktop.test4450.txt'"
    String igorCmd
    sprintf igorCmd, "do shell script \"%s\"", unixCmd
    Print igorCmd       // For debugging only
    ExecuteScriptText/b/UNQ/W=1/Z igorCmd
    Print S_value       // For debugging only
    return S_value
    return "s"
End


Function/S DemoUnixShellCommand2()
    String unixCmd
    String igorCmd
    unixCmd = "killall nc"
    sprintf igorCmd, "do shell script \"%s\"", unixCmd
    Print igorCmd       // For debugging only
    ExecuteScriptText/b/UNQ/W=1/z igorCmd
    Print S_value       // For debugging only
    return S_value
    return "s"
End




Function TestTask(s)        // This is the function that will be called periodically
    STRUCT WMBackgroundStruct &s
DemoUnixShellCommand()
DemoUnixShellCommand2()
return 0    // Continue background task
End





Function StartTestTask()
    Variable numTicks = 2 * 60      // Run every two seconds (120 ticks)
    CtrlNamedBackground Test, period=numTicks, proc=TestTask
    CtrlNamedBackground Test, start
End

Function StopTestTask()
    CtrlNamedBackground Test, stop
End

 

I have no experience with ExecuteScriptText, but in the manual the following is written:

For non-GUI programs, if you specify a positive value for waitTime, ExecuteScriptText waits up to that many seconds for the command to complete. If the command fails to complete within that period of time, ExecuteScriptText returns an error. If you omit the /W flag or if you pass zero for waitTime, ExecuteScriptText waits until the command completes no matter how long it takes. If the script being executed creates a Windows command window, via cmd.exe for example, the Windows process completes only when the command window closes, not after the script commands execute.

 Maybe you have to make sure that the shell script closes the command window via exit or something?

In reply to by silvan.roth

silvan.roth wrote:

yes, but how can I avoid pressing Ctrl-Break or Shift-Esc, since I want this to run continuous, picking up the value in the txt file every second?

Based on the paths in your code snippet I assume you are running on macOS. As documented in the ExecuteScriptText command help, the /W flag is accepted but ignored on Macintosh. So you can't tell Igor to use a timeout on macOS.

 

I think you can get this to work if you use these two functions to start and stop listening:

Function StartListening()
    String ncCommand = "nc -l -u 4450 &> '/Users/srot/desktop/test4453.txt'"
   
#ifdef WINDOWS
    // NOT TESTED!!!!!!
    ExecuteScriptText/Z /W=1 "cmd.exe /K " + ncCommand
#else
    String cmd="tell application \"Terminal\"\ractivate\rdo script \""
    cmd+=ncCommand+"\"\r"
    cmd+="\rend tell\r"
    ExecuteScriptText/UNQ cmd
#endif
End


Function/S StopListening()
    String unixCmd
    String igorCmd
    unixCmd = "killall nc"
    sprintf igorCmd, "do shell script \"%s\"", unixCmd
    Print igorCmd       // For debugging only
    ExecuteScriptText/b/UNQ/W=1/z igorCmd
    Print S_value       // For debugging only
    return S_value
    return "s"
End

 

You would then start a background task that polls the file. You probably want to keep a global variable that stores the number of bytes in the file that you have already read/processed. In your background task you would open the file, set the position to the previous end position, read the rest of the file, store the new end position, and close the file. Look at the command help for Open, FBinRead, FStatus, and FSetPos.

Well, the code works, but only once, after that I have to send another set of StartListening() and StopListening(). If I now put this in a Loop, it completely spams me with Terminal windows. 
Let me explain the whole task: I have a device pushing with about 10Hz values over the UDP port 4450. I want to read these values with Igor and store them in a wave. Since I cannot directly read UDP with Igor I thought of going via a txt file, constantly refreshing it. Another possible way I see is forwarding the UDP port to a TCP port. Igor should be able to read from that

In reply to by silvan.roth

silvan.roth wrote:

Well, the code works, but only once, after that I have to send another set of StartListening() and StopListening(). If I now put this in a Loop, it completely spams me with Terminal windows. 
Let me explain the whole task: I have a device pushing with about 10Hz values over the UDP port 4450. I want to read these values with Igor and store them in a wave. Since I cannot directly read UDP with Igor I thought of going via a txt file, constantly refreshing it. Another possible way I see is forwarding the UDP port to a TCP port. Igor should be able to read from that

Since your question no longer involves the SOCKIT XOP, it's probably best if you create a new thread to continue discussion.

However at this point, I think you have a UNIX problem, not an Igor problem. If you execute the following in terminal, do you get a continuous supply of messages, or just one message?

nc -l -u 4450

If you only get one message, you need to figure out a command that gives you a continuous stream of messages. Here is a command that might give you something closer to what you want:

sudo tcpdump -u -A -t port 4450

That will print the header and some other information that you probably don't want, but with sufficient UNIX fu you should be able to get all that stripped off. https://serverfault.com/questions/206734/how-do-i-make-tcpdump-not-prin… gives some suggestions that might work. Otherwise you can just dump all of it to the text file and extract the data itself in Igor.

The way I've usually dealt with UDP data is to use python scripts to send/receive the data streams, write to a text file, and read the text file periodically with Igor as a workaround. I don't know if this is a possible solution in your case.

In reply to by kaikin

When applying the code above, a ew terminal is opened every time the command is called. Furthermore, you need to execute it as sudo, which means that for every datapoint you would have to enter the pwd. If you modify the code like this:

Function LoopDump()
String ncCommand = "sudo tcpdump -u -A -t port 4450 -c 1 -w logg.txt"
   String cmd="tell application \"Terminal\"\ractivate\rdo script \""
   
    cmd+=ncCommand
    cmd+="\" "
  cmd+=" in window 1"
    cmd+="\rend tell\r"
   print cmd
  ExecuteScriptText/UNQ cmd
end

it resends the command to the same terminal again and again. 
Unfortunately, the output txt file is not readable by Igor as a notebook. Any Idea?

Output txt file

If you got to the point where a text file is constantly updated, and the remaining task is 'only' to watch this file and read its contents, I think you are almost there. Igor can do this easily. I guess the first task is to build a function which can parse the file contents. I guess the part you are interested in is the binary data after tcpdump, right? Do you know what to expect and how to parse the data? Look into FBinRead for loading the the binary data. If you got this part down, then you could write a background task which checks upon the file and load the latest output. Here is some pseudocode:

static Function/S readLog(String fullPathToFile)
    int FileID
    String read = ""   
    Open/Z/R fileID as fullPathToFile
    if (!V_flag)
        FBinRead fileID, read // find the correct settings
        Close fileID
    endif
    return read
End

Oh, that's easy. Try this (assuming the value you are interested in is prepended by '|' and ends with ';'... if multiple such entries are contained it gets a bit more complicated):

Function readLog(String fullPathToFile)
    int FileID
    String read = "", output = ""  
    Open/Z/R fileID as fullPathToFile
    if (!V_flag)
        do
            FReadLine fileID, read
        output += read
        while (strlen(read) > 0)
        Close fileID
    endif
    int start = strsearch(output,"|",0) + strlen("|")
    int stop  = strsearch(output,";",start)- strlen(";")
    return str2num(output[start,stop])
End

As a test, execute: print/D readLog("thisStringIsAFilePath")

In reply to by chozo

it works! thanks a lot!

However, it's really slow, since I have to wait for the file to be passed through UDPdump and being saved. I think there is no way around writing my own XOP for reading UDP values. Do you have a starting point for me?

Why are you using the -c flag with tcpdump? If you remove that flag then you shouldn't need to call the tcpdump command over and over.

Using the -w flag writes raw packets to file using the pcap file format. This is a binary file format, and doing what chozo suggested above might work but is risky.

Try this command:

sudo tcpdump -u -A -t -l port 4450 > ~/Desktop/tcp.txt

When I use this command on my machine that has a device that sends UDP packets at around 1Hz, I see the size of the tcp.txt output file increase about once a second. The -l flag should force tcpdump to write the data to the file one line at a time, which is probably what you want. Polling for changes to that file's size from Igor and reading in the new data should easily be able to keep up with a 10 Hz device.

That works well, and the file is growing. If I now keep reloading the file, it will be soon slow. Can I delete the loaded lines in the file with Igor again?

In reply to by silvan.roth

silvan.roth wrote:

That works well, and the file is growing. If I now keep reloading the file, it will be soon slow. Can I delete the loaded lines in the file with Igor again?

Writing to a file from two different processes is a bad idea, if it's even possible.

I figured out how to get nc to work to capture multiple UDP packets to a file:

while true; do printf '\r' | nc -lu 50222 >> ~/Desktop/nc.txt; done

Depending on how long you capture data, that file will eventually get large. I'm not sure that will actually cause performance problems (unless it gets **really** large). You may need to capture to a different file periodically so you can finish reading the previous file and delete it.

Another approach you could consider (and possibly quickly reject) is to direct the output of nc directly to Igor. To test this, I pasted the following function into Igor's procedure window and compiled procedures.

Function RecordUDPPacket(String packet)
    WAVE/Z/T packetWave = root:packetWave
    if (!WaveExists(packetWave))
        Make/O/N=0/T root:packetWave
        WAVE/T packetWave = root:packetWave
    endif
   
    packetWave[numpnts(packetWave)] = {packet}
End

I then executed this command in terminal:

while true; do printf '\r' | nc -lu 50222 | xargs -I{} /Applications/Igor\ Pro\ 9\ Folder/Igor64.app/Contents/MacOS/Igor64 /X RecordUDPPacket\(\""{}"\"\) ; done

This attempts to start a new Igor instance each time it is called, but since one is already running that instance takes the command. The device I'm using sends packets at around 1 Hz. Your 10Hz device might bog down the OS too much. Also, Igor prints the command into the history window and it's not clear to me that it's possible to prevent that from happening.

BTW, I'm testing this with a device that uses port 50222. You should substitute your port number.

Writing your own XOP might be an option if you are an experienced C/C++ programmer and if you are familiar with the UDP protocol.

You might be able to use the ZeroMQ XOP (https://github.com/AllenInstitute/ZeroMQ-XOP) to listen for UDP packets on a port. I don't know much about ZeroMQ and have never used the XOP.

In reply to by aclight

it almost works sending the packets to Igor directly. When I paste the command in the terminal, nothing happens, the computer stays in listening mode. I close the terminal manually and reopen a new terminal and send the command killall nc. As soon as I send this, Igor receives a package.