Recent Changes for "Processing Software" - MACCS Magnetometer Array

Processing Software

maccs — 2007-09-04 16:51:06

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 53:	Line 53:
- [http://space/space/index.html index.html]	+ [http://space/space/index.html]

Processing Software

maccs — 2007-09-04 16:50:38

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 53:	Line 53:
- ["index.html"]	+ [http://space/space/index.html index.html]

Processing Software

maccs — 2007-09-04 16:47:33

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 51:	Line 51:
- =Augsburg WebServer Programs=	+ =Augsburg WebServer Date Request Application=
Line 54:	Line 54:
- This is the webpage for the space/space MACCS data interactive request form. This site can be used to obtain binary data, IAGA ASCII data, png plots, or postscript plots of this data. The Data Request Application is created by the perl script datareq which calls the idl program datareq.pro upon form submission. Location: /Library/WebServer/Documents/space/.	+ This is the web page for the space/space MACCS data interactive request form. This site can be used to obtain binary data, IAGA ASCII data, png plots, or postscript plots of this data. The Data Request Application is created by the perl script datareq which calls the idl program datareq.pro upon form submission. Location: /Library/WebServer/Documents/space/.

Processing Software

maccs — 2007-09-04 16:45:07

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 51:	Line 51:
	+ =Augsburg WebServer Programs= + + ["index.html"] + This is the webpage for the space/space MACCS data interactive request form. This site can be used to obtain binary data, IAGA ASCII data, png plots, or postscript plots of this data. The Data Request Application is created by the perl script datareq which calls the idl program datareq.pro upon form submission. Location: /Library/WebServer/Documents/space/. + + ["datareq"] + This is the perl script that generates the Data Request Application web page. It calls datareq.pro upon sumbission of the form. Location: /Library/WebServer/CGI-Executables/maccs/ + + ["datareq.pro"] + This program is called by datareq to display the requested data or graphs on the website. Location: /Library/WebServer/CGI-Executables/maccs/ +

Processing Software

maccs — 2007-09-04 16:04:26

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 1:	Line 1:
- Describe Processing Software here.	+ =MACCS Iridium Data Processing At Augsburg=
Line 3:	Line 3:
- MACCS Iridium Data Processing At Augsburg	+ ==Crontab Scripts & Programs==
Line 5:	Line 5:
- The shell script used to download and process the raw data files - ["ftpsri.sh"] - The idl program used to make Augsburg format ASCII files - ["IridiumMaccs2.pro"] - The idl program used to make IAGA2000 format ASCII files - ["IridiumMaccs3.pro"] - The idl program used to make line plots - ["linex7.pro"]	+ Listing of maccs user crontab:
Line 14:	Line 7:
- 2004 Boston University MACCS Data Processing	+ 0,15,30,45 * * * * /Volumes/physics_data/Code/MACCS_Code/ftpsri.sh + 35 19 * * * /Volumes/physics_data/Code/MACCS_Code/checkMACCSir.sh + + ["ftpsri.sh"] + This program runs at :00, :15, :30, :45 of every hour of every day. It downloads the latest MACCS iridium raw data files from SRI's ftp site and files them into the MACCS_DATA/Raw/ folder. Then it makes ASCII files and line plots of the data it downloaded. It calls the following idl programs (from the /Volumes/physics_data/Code/MACCS_Code/ directory) to process the data: IridiumMaccs2.pro to make Augsburg Format ASCII files which are used by linex7.pro, IridiumMaccs3.pro to make IAGA2000 Format ASCII files which are made available on the MACCS iridium website, and linex7.pro to make line plots of the data which are made available on the MACCS iridium website. It finishes by running the todaysdata.sh shell script which make copies of the line plots for the Today's Real-Time Data Plots web page. + + ["IridiumMaccs3.pro"] + This program is run by ftpsri.sh at :00, :15, :30, and :45 past every hour of every day. It creates IAGA2000 format ASCII files from the raw iridium data. + + ["IridiumMaccs2.pro"] + This program is run by ftpsri.sh at :00, :15, :30, and :45 past every hour of every day. It makes Augsburg format ASCII files from the raw iridium data. + + ["linex7.pro"] + This program is run by ftpsri.sh at :00, :15, :30, and :45 past every hour of every day. It run on the Augsburg format ASCII files created by IridiumMaccs2.pro. It creates line graphs of the iridium data. + + ["todaysdata.sh"] + This script is run by ftpsri.sh at :00, :15, :30, :45 past every hour of every day. It copies the current day's line plot files to the web server and renames them. These files are made available on the Today's Real-Time Data Plots web page. + + ["checkMACCSir.sh"] + This script runs at 19:35. It emails the newest MACCS Iridium data line plot files to the listed users. If there is no new data it sends an email to report the missing data. + + ==Non-Crontab Scripts== + + ["badCD.sh"] + This script should be run on days when there is a problem with the Cape Dorset data. It moves the current day's raw and processed CD files to the Problem_Data/ folders. It can also be used to move other day's files to the Problem_Data/ folders. If the new data has just gone bad, uncomment the last section of the script to add a warning to the Today's Real-Time Data Plots web page. + + ["goodCDtoday.sh"] + This script should be run the UTC day after the problem with the Cape Dorset data has been fixed. It removes the warning from the Today's Real-Time Data Plots web page. + + ["badNA.sh"] + This script should be run on days when there is a problem with the Nain data. It moves the current day's raw and processed NA files to the Problem_Data/ folders. It can also be used to move other day's files to the Problem_Data/ folders. If the new data has just gone bad, uncomment the last section of the script to add a warning to the Today's Real-Time Data Plots web page. + + ["goodNAtoday.sh"] + This script should be run the UTC day after the problem with the Nain data has been fixed. It removes the warning from the Today's Real-Time Data Plots web page. + + ["maccsprocessing.sh"] + Convert the CD and NA iridium data into ASCII files and line plots a year at a time. + + ["processCD.sh"] + This script can be used to process a single raw Cape Dorset data file. The defult day is the current UTC day. + + ["processNA.sh"] + This script can be used to process a single raw Nain data file. The defult day is the current UTC day. + + + =2004 Boston University MACCS Data Processing=

Processing Software

maccs — 2007-08-21 15:47:26

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 14:	Line 14:
- Boston University MACCS Data Processing	+ 2004 Boston University MACCS Data Processing

Processing Software

maccs — 2007-08-20 19:28:10

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 2:	Line 2:
	+ + MACCS Iridium Data Processing At Augsburg + + The shell script used to download and process the raw data files + ["ftpsri.sh"] + The idl program used to make Augsburg format ASCII files + ["IridiumMaccs2.pro"] + The idl program used to make IAGA2000 format ASCII files + ["IridiumMaccs3.pro"] + The idl program used to make line plots + ["linex7.pro"] + + Boston University MACCS Data Processing

Processing Software

maccs — 2007-08-06 20:40:44

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 7:	Line 7:
- ["igloolik_despike.pro"]	+ ["igloolik despike.pro" igloolik_despike.pro]

Processing Software

maccs — 2007-08-06 20:40:17

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 9:	Line 9:
- ["write_maccs.pro"] (called by igllolik_despike.pro)	+ ["write maccs.pro" write_maccs.pro] (called by igllolik_despike.pro)
Line 11:	Line 11:
- ["step_view.pro"] - ["step_wid.pro"]	+ ["step view.pro" step_view.pro] + ["step wid.pro" step_wid.pro]
Line 15:	Line 15:
- ["five_sec_avg.pro"] - ["five_sec_avg_fast_avg.pro"] (called by five_sec_avg.pro) - ["read_maccs.pro"] (called by five_sec_avg.pro) - ["write_maccs.pro"] (called by five_sec_avg.pro)	+ ["five sec avg.pro" five_sec_avg.pro] + ["five sec avg fast avg.pro" five_sec_avg_fast_avg.pro] (called by five_sec_avg.pro) + ["read maccs.pro" read_maccs.pro] (called by five_sec_avg.pro) + ["write maccs.pro" write_maccs.pro] (called by five_sec_avg.pro)
Line 20:	Line 20:
- ["sec_to_min.pro"] - ["read_maccs.pro"] (called by sec_to_min.pro) - ["write_maccs.pro"] (called by sec_to_min.pro)	+ ["sec to min.pro" sec_to_min.pro] + ["read maccs.pro" read_maccs.pro] (called by sec_to_min.pro) + ["write maccs.pro" write_maccs.pro] (called by sec_to_min.pro)
Line 25:	Line 25:
- ["merge_files.pro"]	+ ["merge files.pro" merge_files.pro]
Line 33:	Line 33:
- ["hour_read.pro"]	+ ["hour read.pro" hour_read.pro]
Line 35:	Line 35:
- ["wid_draw.pro"]	+ ["wid draw.pro" wid_draw.pro]

Processing Software

maccs — 2007-08-06 20:37:13

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 8:	Line 8:
- ["read_maccs.pro" read maccs.pro] (called by igllolik_despike.pro)	+ ["read maccs.pro" read_maccs.pro] (called by igllolik_despike.pro)

Processing Software

maccs — 2007-08-06 20:36:50

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 8:	Line 8:
- ["read_maccs.pro"] (called by igllolik_despike.pro)	+ ["read_maccs.pro" read maccs.pro] (called by igllolik_despike.pro)

Processing Software

maccs — 2007-07-25 21:42:43

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 26:	Line 26:
- Nb. Requires an extra file "["missing.1m"]", to substitute	+ Nb. Requires an extra file "[[File(missing.1m)]]", to substitute

Processing Software

maccs — 2007-07-25 21:42:21

Upload of file missing.1m.

Processing Software

maccs — 2007-07-25 21:24:50

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 11:	Line 11:
- step_view.pro - step_wid.pro - flag.pro	+ ["step_view.pro"] + ["step_wid.pro"] + ["flag.pro"]
Line 15:	Line 15:
- five_sec_avg.pro - five_sec_avg_fast_avg.pro (called by five_sec_avg.pro)	+ ["five_sec_avg.pro"] + ["five_sec_avg_fast_avg.pro"] (called by five_sec_avg.pro)
Line 20:	Line 20:
- sec_to_min.pro	+ ["sec_to_min.pro"]
Line 24:	Line 24:
- expandcode.pro - merge_files.pro - Nb. Requires an extra file "missing.1m", to substitute	+ ["expandcode.pro"] + ["merge_files.pro"] + Nb. Requires an extra file "["missing.1m"]", to substitute
Line 29:	Line 29:
- houravrg.pro	+ ["houravrg.pro"]
Line 31:	Line 31:
- findbeta.pro - strip.pro: - hour_read.pro - flag.pro - wid_draw.pro	+ ["findbeta.pro"] + ["strip.pro"] + ["hour_read.pro"] + ["flag.pro"] + ["wid_draw.pro"]

Processing Software

maccs — 2007-07-25 21:16:53

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 7:	Line 7:
- igloolik_despike.pro - read_maccs.pro (called by igllolik_despike.pro) - write_maccs.pro (called by igllolik_despike.pro)	+ ["igloolik_despike.pro"] + ["read_maccs.pro"] (called by igllolik_despike.pro) + ["write_maccs.pro"] (called by igllolik_despike.pro)
Line 17:	Line 17:
- read_maccs.pro (called by five_sec_avg.pro) - write_maccs.pro (called by five_sec_avg.pro)	+ ["read_maccs.pro"] (called by five_sec_avg.pro) + ["write_maccs.pro"] (called by five_sec_avg.pro)
Line 21:	Line 21:
- read_maccs.pro (called by sec_to_min.pro) - write_maccs.pro (called by sec_to_min.pro)	+ ["read_maccs.pro"] (called by sec_to_min.pro) + ["write_maccs.pro"] (called by sec_to_min.pro)

Processing Software

maccs — 2007-07-25 21:13:45

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 5:	Line 5:
- [[File(strip.pro)]] (called by cleaner.pro)	+ ["strip.pro"] (called by cleaner.pro)

Processing Software

maccs — 2007-07-25 21:11:37

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 5:	Line 5:
- strip.pro (called by cleaner.pro)	+ [[File(strip.pro)]] (called by cleaner.pro)

Processing Software

maccs — 2007-07-25 21:10:58

Upload of file strip.pro.

Processing Software

maccs — 2007-07-25 21:09:58

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 3:	Line 3:
- *The IDL program used in the clean step	+ *The IDL programs used in the clean step
Line 6:	Line 6:
- *The IDL program used in the despike step	+ *The IDL programs used in the despike step
Line 10:	Line 10:
- *The IDL program used in the manual step of removing spikes	+ *The IDL programs used in the manual step of removing spikes
Line 14:	Line 14:
- *The IDL program used in the 1/2 sec to 5 sec avg step	+ *The IDL programs used in the 1/2 sec to 5 sec avg step
Line 19:	Line 19:
- *The IDL program used in the 5sec to 1 min avg step	+ *The IDL programs used in the 5sec to 1 min avg step
Line 23:	Line 23:
- *The IDL program used to merge all 1 min files in 1 big file	+ *The IDL programs used to merge all 1 min files in 1 big file
Line 28:	Line 28:
- *The IDL program used in the 1 min merged files to 1 hour avg step	+ *The IDL programs used in the 1 min merged files to 1 hour avg step
Line 30:	Line 30:
- *The IDL program used in prepare the baselines for rotation	+ *The IDL programs used in prepare the baselines for rotation
Line 36:	Line 36:
- *The IDL program used in the rotate step	+ *The IDL programs used in the rotate step

Processing Software

maccs — 2007-07-25 21:04:45

Edited By Erik Johnson 7/25/07

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 5:	Line 5:
- strip.pro	+ strip.pro (called by cleaner.pro)
Line 8:	Line 8:
- read_maccs.pro - write_maccs.pro	+ read_maccs.pro (called by igllolik_despike.pro) + write_maccs.pro (called by igllolik_despike.pro)
Line 15:	Line 15:
- five_sec_avg_fast_avg.pro
Line 17:	Line 16:
	+ five_sec_avg_fast_avg.pro (called by five_sec_avg.pro) + read_maccs.pro (called by five_sec_avg.pro) + write_maccs.pro (called by five_sec_avg.pro)
Line 18:	Line 20:
- sec_to_min_fast_avg.pro
Line 20:	Line 21:
	+ read_maccs.pro (called by sec_to_min.pro) + write_maccs.pro (called by sec_to_min.pro)
Line 25:	Line 28:
- The IDL program used in the 1 min merged files to 1 hour avg step	+ *The IDL program used in the 1 min merged files to 1 hour avg step
Line 34:	Line 37:
	+ ?

Processing Software

maccs — 2007-07-24 21:09:40

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 4:	Line 4:
- [cleaner.pro]	+ ["cleaner.pro"]

Processing Software

maccs — 2007-07-24 21:09:26

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 4:	Line 4:
- cleaner.pro	+ [cleaner.pro]

Processing Software

maccs — 2007-07-24 21:08:00

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 7:	Line 7:
- igloolik_despike.pro - read_maccs.pro - write_maccs.pro - The IDL program used in the manual step of removing spikes - step_view.pro - step_wid.pro - flag.pro - The IDL program used in the 1/2 sec to 5 sec avg step - five_sec_avg_fast_avg.pro - five_sec_avg.pro - The IDL program used in the 5sec to 1 min avg step - sec_to_min_fast_avg.pro - sec_to_min.pro - The IDL program used to merge all 1 min files in 1 big file - expandcode.pro - merge_files.pro - Nb. Requires an extra file "missing.1m", to substitute - missing days	+ igloolik_despike.pro + read_maccs.pro + write_maccs.pro + The IDL program used in the manual step of removing spikes + step_view.pro + step_wid.pro + flag.pro + The IDL program used in the 1/2 sec to 5 sec avg step + five_sec_avg_fast_avg.pro + five_sec_avg.pro + The IDL program used in the 5sec to 1 min avg step + sec_to_min_fast_avg.pro + sec_to_min.pro + The IDL program used to merge all 1 min files in 1 big file + expandcode.pro + merge_files.pro + Nb. Requires an extra file "missing.1m", to substitute + missing days
Line 26:	Line 26:
- houravrg.pro - The IDL program used in prepare the baselines for rotation - findbeta.pro - strip.pro: - hour_read.pro - flag.pro - wid_draw.pro - The IDL program used in the rotate step	+ houravrg.pro + The IDL program used in prepare the baselines for rotation + findbeta.pro + strip.pro: + hour_read.pro + flag.pro + wid_draw.pro + The IDL program used in the rotate step

Processing Software

maccs — 2007-07-24 21:06:46

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 4:	Line 4:
- cleaner.pro - strip.pro	+ cleaner.pro + strip.pro

Processing Software

maccs — 2007-07-24 21:06:14

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 3:	Line 3:
- 1. The IDL program used in the clean step	+ *The IDL program used in the clean step
Line 6:	Line 6:
- 2. The IDL program used in the despike step	+ *The IDL program used in the despike step

Processing Software

maccs — 2007-07-24 21:05:31

Differences for Processing Software

Deletions are marked with - .	Additions are marked with +.
Line 1:	Line 1:
	+ Describe Processing Software here. + + 1. The IDL program used in the clean step + cleaner.pro + strip.pro + 2. The IDL program used in the despike step + igloolik_despike.pro + read_maccs.pro + write_maccs.pro + The IDL program used in the manual step of removing spikes + step_view.pro + step_wid.pro + flag.pro + The IDL program used in the 1/2 sec to 5 sec avg step + five_sec_avg_fast_avg.pro + five_sec_avg.pro + The IDL program used in the 5sec to 1 min avg step + sec_to_min_fast_avg.pro + sec_to_min.pro + The IDL program used to merge all 1 min files in 1 big file + expandcode.pro + merge_files.pro + Nb. Requires an extra file "missing.1m", to substitute + missing days + The IDL program used in the 1 min merged files to 1 hour avg step + houravrg.pro + The IDL program used in prepare the baselines for rotation + findbeta.pro + strip.pro: + hour_read.pro + flag.pro + wid_draw.pro + The IDL program used in the rotate step