I did not find anything within the existing forum so hopefully the community will have some ideas.
Here's my situation. I have a CR1000 with the Std.24 OS. My client has all loggers on Std.24 so options to upgrade to 26 are limited. This logger measures a set of temperatures and uses the HTTPPost command to send this data via a formatted URL string to an API on a remote server that inserts the data into a database. When running normally (nobody connected to the logger) the watchdog triggers about every 7 minutes and data is not sent to the server for 75 seconds (the TCP timeout duration, I think). The status table shows NO missed scans, only watchdog errors. However, if anyone connects to the logger via Loggernet, regardless of where they connect from (server, laptop, data center, etc), the watchdog does not trigger and there are no missed posts to the API.
The note in the status table is:
"Compile results for last program sent: Warning: Watchdog timer for IP Panic triggered."
The program is compiled in PipelineMode.
Can anyone explain how connecting from anywhere via Loggernet resolves the HTTP post timeout? The same issue happens if I use HTTPGet too. Suggestions? Thanks!
After digging through more historical data, the interval between events is more random and varies from between 2 and 7 minutes. The loss of data (timeout?) is either 60 or 75 seconds, granted the scan interval is 15 seconds.
Ryan,
I know it's not the desired approach, but really, upgrading (at least one) of the OSes would be the first place to start. Several conditions that triggered watchdogs over IP were resolved in OS 25 and 26 (you can check the Revision history for the OS on the web site for these issues). At least one of the issues was not really an error in the datalogger, but just the result of a timeout period (which was subsequently adjusted).
Dana W.
Dana,
I updated the logger to OS 26 this morning. I no longer see the IP Panic watchdog, but now see a more consistent problem. With the HTTPGet command within a 15-second scan, only every third scan is successful and sent to the API. Based on record numbers it is waiting through two scan intervals. Lots of skipped scans in the status table. So, it now seems that a timeout in HTTPGet is dominating. However, this is a time that I cannot adjust.
I've tried going back to the old TCPOpen, SerialOut, SerialIn, TCPClose method of sending the data since I can adjust those timeouts. But, no luck as of yet. The socket opens correctly (value > 100) but I'm apparently not sending the correct header as of yet.
I am not the best one on these HTTP problems, but one thing I would look at is putting HTTPGet in a slow sequence. That way, the datalogger will run the main scan and the HTTPGet task will be run based on the time allowed (when the main scan isn't running).
I *think* you may also be able to use the IPTrace capability of the datalogger to monitor the packets, which may (or may not) help to give you some info into what might be happening.
In DevConfig, connect to the datalogger (or choose File | Off-line mode for a cr1000), and look at the the Settings Editor, IPTrace Code help. Set up the COM port to send the trace info to, and then monitor that port (if you have physical access and can actually do that....).
Dana W.
The thought crossed my mind. But, the scan interval of the main program is the rate at which I need to write to the API. If I place the HTTPGet into a slow sequence then I likely will have to accept a reduced set of records making it to the API, right?
I have not had to keep track of which records in a table have been sent before. What is the best method to make sure that all records get sent when the slow sequence is at a fractional rate of the main scan?
So... have you tried using TCPOpen to keep the socket open but do the task with HTTPost (instead of trying to use the serial commands)?
One thing to consider with the latest OS (26) is that optional parameters have been added that allow you to "stream" the data when using TableFile, FTPClient, EmailSend, HTTPPost, and HTTPPut.
The following is from the most recent version of the CRBasic help file (which may or may not be in the help that you downloaded with the OS -- I'm not certain.) Sorry for the poorly formatted, volume of info, but if you don't already have this info, it may be useful.
Optional
NumRecs/TimeIntoInterval
The NumRecs parameter is used only when streaming data directly from a data table or data table field. If Interval is greater than 0, the NumRecs/TimeIntoInterval parameter specifies the time into the interval previously unsent records should be written to file on the server.
If Interval is equal to 0, the NumRecs/TimeIntoInterval parameters specifies the number of previously unsent records that will be written to file on the server. If Interval is equal to 0, a negative NumRecs/TimeIntoInterval parameter will specify the number of records that will be written to file on the server each time the function is called.
The datalogger keeps track of the last record sent in the data table information. This information is maintained even through power-downs, as long at the data table itself in datalogger memory is intact.
Interval The Interval parameter is used only when streaming data directly from a data table or data table field. If greater than 0, the Interval parameter determines the interval at which previously unsent data will be written to file. If equal to zero, the NumRecs parameter will control when data is written. A negative Interval will cause the datalogger to write an Interval’s worth of data each time the function is called.
NumRecs/TimeIntoInterval Interval Sent
>=0>0
Interval’s worth of previously unsent records. Only previously unsent data will be sent. If multiple intervals have passed between calls to this function, multiple files will be written to the server and each file will contain Interval’s worth of records.
>=0
0
Number of previously unsent record. Only previously unsent data will be sent. If multiples of NumRecs have been recorded between calls to this function, multiple files will be written to the server and each file will contain NumRecs of records.
<0
0
Each time this function is called, NumRecs records will be sent.
0
<0
Each time this function is called, Interval’s worth of records will be sent.
Units The Units parameter is used only when streaming data directly from a data table or data table field. It is used to specify the units on which the TimeIntoInterval and Interval parameters will be based. The options are microseconds, milliseconds, seconds, minutes, hours, or days.
FileOption The FileOption parameter is used only when streaming data directly from a data table or data table field. It specifies the format used when writing data to the server. The file created on the server will automatically be appended with an incrementing file number and a “.dat” file extension. If 1000 is added to the format (e.g., 1008), the datalogger will not automatically append the incrementing number or “.dat” extension to the uploaded file; this allows the user to have full control of what, if anything, is appended to the file name. Options 0, 8, 16, and 20 correspond to CSI's defined formats for TOB1, TOA5, CSIXML, and CSIJSON, respectively.
Code Description
0 TOB1, Header, TimeStamp, Record#
1 TOB1, Header, TimeStamp
2 TOB1, Header, Record#
3 TOB1, Header
4 TOB1, TimeStamp, Record#
5 TOB1, TimeStamp
6 TOB1, Record#
7 TOB1
8 TOA5, Header, TimeStamp, Record#
9 TOA5, Header, TimeStamp
10 TOA5, Header, Record#
11 TOA5, Header
12 TOA5, TimeStamp, Record#
13 TOA5, TimeStamp
14 TOA5, Record#
15 TOA5
16 CSIXML, TimeStamp, Record#
17 CSIXML, TimeStamp
18 CSIXML, Record#
19 CSIXML
32 CSIJSON, TimeStamp, Record#
33 CSIJSON, TimeStamp
34 CSIJSON, Record#
35 CSIJSON
Thanks for the suggestion. I did try using TCPOpen to define the socket and they used HTTPPost (or HTTPGet) to write to the API. In both cases the timeout issue occurred. I also tried closing the socket after the data was sent. Same response, although I could see the value of the socket increase each interval.
I did see the additional options in HTTPost, but I'm not sure how they will help in my situation. I did not find any examples of how to use this new functionality. Is there example code that you can post? How would streamed data be received by the server?
I've not personally used the streaming with HTTPPost (did I mention I'm not the best one to help with this stuff? :)).
One of the guys who has worked a lot with the instruction is out this week. Perhaps he will jump in and provide some suggestions that will help when he returns.
Dana W.