Trace:

internal:administration:idl

This shows you the differences between two versions of the page.

Both sides previous revision Previous revision Next revision | Previous revision | ||

internal:administration:idl [2019/10/10 17:11] jan [NETCDF] |
internal:administration:idl [2020/02/24 12:54] (current) jan |
||
---|---|---|---|

Line 656: | Line 656: | ||

s_kurt = m[3] | s_kurt = m[3] | ||

- | With version 8 idl can restrict moment and mean to (only) one dimension | + | With version 8 idl can restrict moment and mean to (only) one dimension: |

- | m = moment( x, dimension = 1 , /nan ) | + | ; generate some data |

+ | N = 1000 | ||

+ | x = randomu(rs,N) ; this gives a fltarr(N) | ||

+ | y = (randomu(rs,N))^2; this gives a fltarr(N) | ||

+ | xy = [ [x], [y] ] ; this gives a fltarr(N,2) | ||

+ | m_x_y = mean( xy, dimension = 1 ) ; this gives a fltarr(2) equal to [ mean(x), mean(y) ] | ||

+ | m_xy = mean( xy, dimension = 2 ) ; this gives a fltarr(N) equal to m_xy[i] = mean([ x[i], y[i] ]) | ||

+ | mom = moment( xy, dimension = 1 ) ; this gives a fltarr(2,4) equal to [ [moment(x)], [moment(y)] ] | ||

with variable dimension indicating the number of the dimension over which to average. The first dimension has number 1. | with variable dimension indicating the number of the dimension over which to average. The first dimension has number 1. | ||

+ | The same can be done with the max and min functions. | ||

+ | max_x_y = max( xy, dimension=1 ) ; this gives a fltarr(2) equal to [ max(x), max(y) ] | ||

+ | max_xy = max( xy, dimension=2 ) ; this gives a fltarr(N) equal to max_xy[i] = max([x[i], y[i]]) | ||

+ | ... | ||

===== speed ===== | ===== speed ===== | ||

Line 666: | Line 676: | ||

IDL becomes slow if you use loops like for, while or repeat and conditional statments like if or case and even slower if you combine them. | IDL becomes slow if you use loops like for, while or repeat and conditional statments like if or case and even slower if you combine them. | ||

- | You can speed up your code if you use idl routines, matrix operations etc. | + | This webpage provides some tips to speed your code: [[http://www.danidl.co.uk/idl.speed.tips.shtml]] |

+ | | ||

+ | You can speed up your code if you use idl routines, matrix operations etc. Some examples below: | ||

==== running average ==== | ==== running average ==== | ||

Line 767: | Line 780: | ||

if N_ts gt 0 then x[i_ts] = !values.f_nan | if N_ts gt 0 then x[i_ts] = !values.f_nan | ||

the if N_ts gt 0 is necessary because i_ts will be equal to -1 if no data is found which will result in an error prior to version IDL 8. But IDL 8 treats negative indices as relative indices - so you might not even notice that something wrong happened. | the if N_ts gt 0 is necessary because i_ts will be equal to -1 if no data is found which will result in an error prior to version IDL 8. But IDL 8 treats negative indices as relative indices - so you might not even notice that something wrong happened. | ||

+ | |||

+ | And similar if you do not want to juggle with NaNs create a new dataset: | ||

+ | |||

+ | i_good = where( x gt threshold, N_good ) | ||

+ | if N_good gt 0 then x_good = x[i_good] $ | ||

+ | else x_good = [] ; this gives !NULL (a pointer pointing nowhere ..) which may result in strange results | ||

+ | |||

===== Programs and Routines ===== | ===== Programs and Routines ===== | ||

Line 888: | Line 908: | ||

open (text)files for **input** into idl: | open (text)files for **input** into idl: | ||

- | s = '' ; this line predefines s as a string and the file is read text line by text line | + | s = '' ; this line predefines s as a string and the file is read textline by textline |

| | ||

OPENR, file_unit, filename, /Get_LUN | OPENR, file_unit, filename, /Get_LUN | ||

... | ... | ||

+ | ; read one textline | ||

READF, file_unit, s | READF, file_unit, s | ||

... | ... | ||

Line 897: | Line 918: | ||

FREE_LUN, file_unit | FREE_LUN, file_unit | ||

- | if the file is compressed with gzip it can be directly read by setting the flag %%/compress%% | + | if the file is compressed with gzip it can be directly read by setting the flag %%/compress%% in %%OPENR%% |

open (text)files for **output** from idl: | open (text)files for **output** from idl: | ||

Line 1323: | Line 1344: | ||

POLAR_CONTOUR, field, az_list, R_list | POLAR_CONTOUR, field, az_list, R_list | ||

+ | |||

+ | ===== ticks ===== | ||

+ | |||

+ | you can define the length of intervals between ticks and the number of sub intervals with {x|y}tickinterval and {x|y}minor where minor gives the number of subintervals - not ticks ! | ||

+ | If you want to have ticks every 2.5 units and minor ticks every 0.1 then: | ||

+ | |||

+ | plot, ... , xtickinterval = 2.5, xminor=5 | ||

+ | |||

+ | or | ||

+ | |||

+ | !X.xtickinterval = 2.5 | ||

+ | !X.minor = 5 | ||

+ | |||

+ | if you do not want minor ticks set | ||

+ | |||

+ | xminor = 1 | ||

+ | | ||

+ | |||

+ | |||

+ | ===== Annotation ===== | ||

+ | |||

+ | you may want to annotate an axis with your own words. | ||

+ | You then have to set {x|y}ticks with the number of intervals (not ticks !) {x|y}tickname with an array of strings for the annotations and {x|y}tickv for the locations where to put the tics. Note that {x|y}ticks is the number of intervals i.e. n_elements({x|y}tickname)-1. If you omit xticks IDL assumes 5 intervals or 6 ticks !. | ||

+ | |||

+ | Here is an example with a logarithmic time axis: | ||

+ | |||

+ | hours_per_day = 24d | ||

+ | days_per_year = 365d | ||

+ | | ||

+ | hour = 1./(hours_per_day*days_per_year) | ||

+ | day = 24*hour | ||

+ | month = 30*day | ||

+ | | ||

+ | plot, [1], [1], $ | ||

+ | xstyle = 1, $ | ||

+ | xrange = [ 0.5 * hour, 2*1e6 ], $ | ||

+ | /xlog, $ | ||

+ | xticks = 9, $ | ||

+ | xtickname = [ '1h', '1d', '1M', '1y', '10y', '100y', '1ky', '10ky', '100ky', '1My' ], $ | ||

+ | xtickv = [ hour, day, month, 1., 10., 100., 1000., 10e3, 100e3, 1e6 ], $ | ||

+ | ystyle = 1, $ | ||

+ | yrange = [ 0.5, 4.5 ], $ | ||

+ | yminor = 1, $ | ||

+ | /nodata | ||

Line 1401: | Line 1466: | ||

10 => Histogram mode. Horizontal and vertical lines connect the plotted points, | 10 => Histogram mode. Horizontal and vertical lines connect the plotted points, | ||

as opposed to the normal method of connecting points with straight lines. | as opposed to the normal method of connecting points with straight lines. | ||

+ | Vertical lines are located in the middle between subsequent x. | ||

See Histogram Mode for an example. | See Histogram Mode for an example. | ||

| | ||

Line 2012: | Line 2078: | ||

You must have [[http://www.ghostscript.com/|ghostscript]] installed, and could also use the ghostscript code for this: | You must have [[http://www.ghostscript.com/|ghostscript]] installed, and could also use the ghostscript code for this: | ||

- | gs -dSAFER -dBATCH -dNOPAUSE -sDEVICE=png16m -r120 -sOutputFile=<filename>.png <ps-filename> | + | gs -dSAFER -dBATCH -dNOPAUSE -sDEVICE=png16m -r120 -dEPSCrop -sOutputFile=<filename>.png <ps-filename> |

==== postscript -> PDF ==== | ==== postscript -> PDF ==== | ||

Line 2227: | Line 2293: | ||

* animation is the only advantage | * animation is the only advantage | ||

+ | |||

+ | ==== polar ==== | ||

+ | |||

+ | The plot procedure can work with polar coordinates - you need to set the /polar keyword | ||

+ | |||

+ | ; define some data points | ||

+ | N = 100 | ||

+ | theta = findgen(N)/N*360. | ||

+ | r_max = 7.4 ;-) | ||

+ | radius = r_max*findgen(N)/N | ||

+ | | ||

+ | plot, radius, theta*!DTOR, $ | ||

+ | /polar, $ | ||

+ | xstyle = 1, $ | ||

+ | xrange = [ -r_max, +r_max], $ | ||

+ | ystyle = 1, $ | ||

+ | yrange = [ -r_max, +r_max], $ | ||

+ | /isotropic | ||

+ | | ||

+ | If you do not fix range and isotropy of the plot it may look weird due to automatic scaling. | ||

+ | |||

+ | You may want to have axes through the center: | ||

+ | |||

+ | plot, radius, theta*!DTOR, $ | ||

+ | /polar, $ | ||

+ | xstyle = 1+4, $ ; exact scaling but invisible (box)axis | ||

+ | xrange = [ -r_max, +r_max], $ | ||

+ | ystyle = 1+4, $ ; exact scaling but invisible (box)axis | ||

+ | yrange = [ -r_max, +r_max], $ | ||

+ | /isotropic | ||

+ | | ||

+ | ; Draw x- and y-axis through 0,0 with ticks to the top or right, respectively. | ||

+ | AXIS, 0, 0, XAXIS=0 | ||

+ | AXIS, 0, 0, YAXIS=0 | ||

+ | |||

+ | You may want to have a radius and angle grid. IDL cant do this for you. You have to do it step by step: | ||

+ | |||

+ | plot, radius, theta*!DTOR, $ | ||

+ | /polar, $ | ||

+ | xstyle = 1+4, $ ; exact scaling and invisible (box)axis | ||

+ | xrange = [ -r_max, +r_max], $ | ||

+ | ystyle = 1+4, $ | ||

+ | yrange = [ -r_max, +r_max], $ | ||

+ | /isotropic | ||

+ | | ||

+ | ; draw and anotate spokes | ||

+ | N_spk = 8 | ||

+ | for i = 0, N_spk-1 do begin | ||

+ | wi = (2*!pi/N_spk)*i | ||

+ | oplot, /polar, [0,90], replicate( !pi/2-wi, 2 ), linestyle=1 | ||

+ | r = 90. | ||

+ | xyouts, r*sin(wi), r*cos(wi), string(wi/!dtor,f='(I0)') | ||

+ | endfor ; i | ||

+ | | ||

+ | ; draw and annotate angle circles | ||

+ | N_w = 181 | ||

+ | w = findgen(N_w)/(N_w-1)*2*!pi | ||

+ | N_r = 3 | ||

+ | for i = 0, N_r-1 do begin | ||

+ | ei = (90.*i)/N_r | ||

+ | oplot, /polar, replicate( 90-ei, N_w), w, linestyle= i gt 0 ? 1 : -1 | ||

+ | xyouts, 0, ei, string(ei,f='(I0)') | ||

+ | endfor ; i | ||

+ | |||

+ | |||

+ | Some routines like XYOUTS and ARROW do not accept the polar keyword (or ignore it). | ||

+ | You may convert your coordinates to cartesian as above or you can use the CV_COORD function to do this: | ||

+ | |||

+ | polar_coo = [[angle],[radius]] | ||

+ | xy_coo = CV_COORD( FROM_POLAR=polar_coo, /DEGREES, /TO_RECT ) | ||

+ | |||

==== contour ==== | ==== contour ==== | ||

Line 2379: | Line 2516: | ||

IDL can draw arrows (or vectors). The basic command is arrow which takes the starting point and end point of a (or several) vectors. It can take coordinates for the data-, normalized- or device system. | IDL can draw arrows (or vectors). The basic command is arrow which takes the starting point and end point of a (or several) vectors. It can take coordinates for the data-, normalized- or device system. | ||

+ | |||

+ | arrow, x0,y0, x1,y1, [ /data | /norm | /device ] [, color=...][, thick=...][...] | ||

+ | |||

+ | Allthough it accepts the /polar keyword it is ignored. If you have vectors in polar coordinates you will need to convert them with cv_coord to cartesian coordinates (=rectangular). | ||

+ | |||

If you have a non isotropic coordinate system with e.g. speed on the x-axis and height on the y-axis you may want plot with reference to the x-axis. This can be done for N vectors (u[i],v[i]) at heights h[i] (i=0,N-1) with: | If you have a non isotropic coordinate system with e.g. speed on the x-axis and height on the y-axis you may want plot with reference to the x-axis. This can be done for N vectors (u[i],v[i]) at heights h[i] (i=0,N-1) with: | ||

Line 2942: | Line 3084: | ||

This way +/-infinity and Nan values are included. IDL sorts them in the following way: | This way +/-infinity and Nan values are included. IDL sorts them in the following way: | ||

data[sort[data]] = [[-inf.values], [finite.values], [+inf.values], [Nan.values] ] | data[sort[data]] = [[-inf.values], [finite.values], [+inf.values], [Nan.values] ] | ||

- | You have to count the respective non-finite values and include it in the index calulation: | + | You have to count the respective non-finite values and include it in the index calculation: |

N_neg_inf = long( total(finite(data,/infinity,sign=-1)) ) | N_neg_inf = long( total(finite(data,/infinity,sign=-1)) ) | ||

- | N_pos_inf = long( total(finite(data,/infinity,sign=-1)) ) | + | N_pos_inf = long( total(finite(data,/infinity,sign=+1)) ) |

N_nan = long( total(finite(data,/nan )) ) | N_nan = long( total(finite(data,/nan )) ) | ||

percentiles = (data[ sort[data] ])[ N_neg_inf + p_fac*(n_elements(data)-1-(N_neg_inf+N_pos_inf+N_nan)) ] | percentiles = (data[ sort[data] ])[ N_neg_inf + p_fac*(n_elements(data)-1-(N_neg_inf+N_pos_inf+N_nan)) ] | ||

An implementation as function can be found in /home/hatpro/idl/lib/f_percentile.pro | An implementation as function can be found in /home/hatpro/idl/lib/f_percentile.pro | ||

- | [[http://en.wikipedia.org/wiki/Percentile|wikipedia]] states that there is no exact method to determine percentiles. | + | There is no exact method to determine percentiles. |

- | Imagine a dataset with 50% of the values lower than y1=-0.1 and the other 50% larger than y2=250. with a large gap from y1 - y2. | + | Imagine a dataset with 50% of the values lower than a vlaue y1 and the other 50% larger than a value y2 with a large gap between y1 and y2. |

- | Where ist the median in this case? | + | * Where ist the median in this case? Is it y1, y2 or somewhere in between? |

- | Is iterpolation between y1 and y2 allowed?. | + | * Is interpolation between y1 and y2 allowed? |

- | Wikipadia gives a method for interpolation between these values... | + | * In the same way any percentile may not be exactly determined |

+ | * [[http://en.wikipedia.org/wiki/Percentile|wikipedia]] gives a method for interpolation between these values... | ||

===== random numbers ===== | ===== random numbers ===== | ||

Line 3399: | Line 3542: | ||

... | ... | ||

- | The line starting with "Users of idl: ..." says that there are 18 license in use meaning that 18/6=3 (i.e. three) instances of idl are running. The line below indicates that user jschween on machine gop is running one instance of idl (using 6 licenses) v8.2 since Nov. 11. | + | The line starting with "Users of idl: ..." says that there are 18 license in use. As one allways needs six licences when running idl this means that 18/6=3 (i.e. three) instances of idl are running. The line below indicates that user jschween on machine gop is running one instance of idl (using 6 licenses) v8.2 since Nov. 11. |

| | ||

If this is missing and/or you get a long strange output the license manager is probably not running. | If this is missing and/or you get a long strange output the license manager is probably not running. | ||

Line 3415: | Line 3558: | ||

password is your password ... | password is your password ... | ||

+ | |||

+ | ===== dead IDL processes ===== | ||

+ | |||

+ | If licencses are obviously blocked by dead idl processes (running for several days, see section about license manager above). You may want to kill them (be careful not to destroy hard work of your colleagues !): | ||

+ | * Login on the respective machine | ||

+ | * see what idl processes are running by executing: | ||

+ | |||

+ | ps -elf | grep idl | ||

+ | |||

+ | * output may look like: | ||

+ | |||

+ | ... | ||

+ | 0 S hatpro 18936 1 0 80 0 - 3513 wait 2019 ? 00:00:00 /bin/bash /usr/local/bin/idl /home/dturner/aerioe/tmp/.run_aerioe.pro | ||

+ | 0 R hatpro 18950 18936 98 80 0 - 85711 - 2019 ? 151-17:57:02 /opt/itt/idl8.2/idl82//bin/bin.linux.x86_64/idl /home/dturner/aerioe/tmp/.run_aerioe.pro | ||

+ | ... | ||

+ | |||

+ | * indicating that user hatpro has an idl job "%%run_aerioe.pro%%" which was started in 2019 and is running for// 151 days// and ~18hours | ||

+ | * its process ID is 18950 and it has been started by process 18936 | ||

+ | * eventually check with the (probable) owner whether this is intended | ||

+ | * to terminate this process do %%kill 18950%% (this is the process ID in this case !) | ||

+ | |||

+ | |||

internal/administration/idl.1570720304.txt.gz · Last modified: 2019/10/10 17:11 by jan

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International