Help with exportdata.html error messages

This page is being developed. 29 Mar 2011.

General Information

popup: "Something went wrong"

If JSOC is functioning, the following known user controlled things may have happened.

• If you entered an explicit record limit and it is for too many records you will see the "something..." message. There is a limit of about 15,000 AIA or 32,000 HMI records allowed for each export request. This is due to memory usage in the query response. We will increase this later. If you got the "something" message after entering a record count, reduce the limit to a smaller number and try again.
• others...

popup: "Requested RecordSet not found or specification error, status=4"

• "The export program failed"
  For protocol=fits, the jsoc_export_as_fits program was called but there were no actual data files in the specified record set. this can happen even when the record count shows one or more records if all of the records do not actually ahve and good data. This is possible since e.g. hmi data has records even for times when no data is available, such as during eclipses. To see if this is the problem you can give the same recordset specification to the lookdata.html program and examine the QUALITY keyword. If QUALITY has the top bit set, i.e. 0x8000000 perhaps with other bits also, then there is no data file for the record to export. We will fix jsoc_export_as_fits to give a non-failed export with simply no files.

  popup: "failed to get series info, Enter correct RecordSet or proceed to processing options"

  This message means that the seriesname part of the RecordSet speciication you entered is not recognized as a valid data series. If you are going to do an hg_patch tracked extracted region you can proceed to the "Processing" drop-down menu and check "hg_patch" where it will generate a list of series for whcih you can specify extracts. If that is not your intent, you may have made a typo in the seriesname or perhaps entered a seriesname that is only accessible via jsoc2.stanford.edu and you are using jsoc.stanford.edu. Jsoc2 is reserved for the HMI and AIA teams for local use or for access to near real time data for space weather forcast research. If you need but do not have a password for jsoc2 access contact an HMI or AIA team member for help. The list of dataseries available is seen in lookdata.html on the first tab. You can check the seriesname spelling there.

  Requested RecordSet not found or specification error, status=4

  This is a general error response from the jsoc_fetch program that exportdata.html talks with. The popup tells you the status and error message. It also tells you the temp file name that contains the stderr error messages from jsoc_fetch. Of course we know you can not get at that file, but if it got far enough to put an actual file name there, and you email that to us, we can see what happened. If the error happened too soon in the processing to have assigned a RequestID or if you are making a "url-quick" export where no RequestId is needed, there will not be an erro log for us to inspect. Some of the error messages and reasons are here:
  • popup: error=Can not open RecordSet 172409 is too many records in one request.
    This error is for the same reason of too many records described above. But in this case the recordset specification you provided implied too many records. You will see the number of records requested in the "Record Count" field. If it is above 15,000 for AIA or 32,000 for HMI, reduce the time range of your request, or get fewer wavelengths at once. This will be increased later.
  • popup: error=RecordSet specified does not exist
    The recordset specification resulted in no records selected, or a parse error was encountered. the first thing to check is the details of the query clauses. There are many ways to make a non-recognized query. One way to check is to verify the recordset spec with lookdata. If hg_patch processing is being used, the query built will be with the '[$]' replaced by the values of t_start and t_stop as: '[t_start-t_stop]' where the actual strings provided are used and not the words. So if there is a systax error in t_start or t_stop the parsing will fail. One recent example was use of ISO format times but omitting the zone part, i.e. 2010-08-03T03:00:00 instead of 2010-08-03T03:00:00Z or 2010-08-03T03:00:00_UTC. Note that if the JSOC time notation is used, 2010.08.03_03 then there is no confusion between the '-' char in the time string and the '-' for the time range. Rule: when using the ISO format always give the Zone field at least in a range given by '-'. If the user explicitly includes a segment name that is not in the series this error will occur.
  more later....

  Export seems to work but export "packing list" not right

  If the export request seemed to go OK and upon asking for Export Status you get an updated page with a table of records, but the Filename field is strange, or empty, or otherwise non-functional, one of the following may have happened.

  Filename has no extension of .fits, .jpg, .mp4, etc.

  The "Filename Format" entry is not verified until it is used by the export program. Thus a typo or error in the string that gets parsed to form a rule to make a filename for each record may fail to make a useful filename. If the final section of the filename is NOT "{segment}" then the storage type standard default extension will not be added. In this case you should add the (usually) ".fits" explicitly. the default will be the name of the segment plus the standard extension for that storage protocol. If the segment name is not helpful pehaps becasue it is too general and you have fashioned the file name structure you want, be sure to add the explicit ".fits".

  .mp4 link that does nothing

  The movie maker program can simply fail to make a movie, in which case sometimes no error message is returned but nothing was made. In this case, the RequestID can be used by JSOC staff to lookup the error logfile and sort out the problem. We will get error return status for the movie and image scripts soon.

  Exportdata.html FAQ

  Some random pointers.

  Export time estimates

  At present the export time to complete estimates are completely ficticious. We just use a default 10 seconds. So all this counter is useful for is to make you think something is not working while repeated status requests tell you it is taking longer and longer. We hope to replace the default with a sensible value based on the export "Method", the "Processing", and the record count.

  Export size estimate

  Export size estimates are made by counting the file sizes of the records requested. If you are asking for processing to be done, e.g. hg_patch, then this size reflects the size of the export implied by the "RecordSet" given BEFORE being processed by the hg_patch program. Thus the size is quite useless in this case. In most other cases the size should be a reasonable estimate of the files, given in MB. Note that if you really like to transport big files and ask for uncompressed FITS files, the actual export will be much larger than this estimate.

  More records returned than expected

  One cause of too many records matched can occur if some of the "prime keys" are not specified. An example might be hmi.M_45s data which has two prime keys, T_REC and CAMERA. T_REC is the time for the record "slot" and CAMERA is which HMI camera was used. The Doppler and LOS field is (so far) always obtained from CAMERA=2 so people sometimes omit the [2] as the second part of the query. I.e. hmi.M_45s[2010.11.03_10:00:16_TAI] will return 2 records:
  • hmi.Ic_45s[2010.11.03_10:00:00_TAI][-2147483648]
  • hmi.Ic_45s[2010.11.03_10:00:00_TAI][2]
  where the first one was probably the result of an error in processing with the internal value of "missing data" in the CAMERA field. The second one with CAMERA=2 contains an OK image. Note that the record with a time closest to the specified time was returned.

  Why are T_REC and T_OBS different

  In general, T_REC is in a uniformly spaced grid of times, so called "slotted" times. T_OBS is the actual average of the shutter-open time for the image tagged at T_REC. Thus T_OBS can differ by up to half of the keyword T_REC_step form T_REC.
  For HMI, T_REC is the target time AT ONE AU. Multiple filtergrams are processed for each "observable" time slot. Each filtergram type (wavelength tuning and polarization) is interpolated in time to the target T_REC as if SDO were at 1AU. T_OBS is the average time of the filtergrams AS OBSERVED. Sampling at 1AU removes an annual sidelobe in oscillation spectra.