read ISD files

[cwhanse]

Add functions to read files from the Integrated Surface Database (ISD) .

One possible approach is to port pvl_getISDdata.m and pvl_readISH.m from PVLib Matlab.

A parser is in rOpenSci at CRAN.

A older python reader could be used, but it is questionable if the code is being maintained. It uses many classes and may not fit well with pvlib's style.

[wholmgren]

Porting from PVLib Matlab would probably be fastest. It looks like NOAA now discourages the ftp access route and recommends the http archive with proper csv files, so the port might end up being fairly independent.

What do you think about putting the code in siphon instead? The ISD is more general purpose than most of the datasets accessible in pvlib.iotools.

I agree with your assessment of the older ish_parser.

[cwhanse]

What do you think about putting the code in siphon instead? The ISD is more general purpose than most of the datasets accessible in pvlib.iotools.

Certainly. Or metpy

I'd rather import the code from a maintained library than maintain it ourselves.

[mikofski]

I would also like to see a reader for NOAA NCDC GSOD, especially for rainfall (see ftp see #668

[nappaillav]

I would like to work on this issue.

So here is my plan to start :
I would first go through the matlab codes 1) pvl_getISDdata.m and 2) pvl_readISH.m
also there is a parser in CRAN's repository.

So what would you recommend me to do? if there is any other information related to this issue please do let me know :)

[cwhanse]

@nappaillav from the discussion above, I think a good first step is asking if siphon or metpy is interested to implement an ISD reader.

[nappaillav]

@cwhanse have look into this eeweather which supports Integrated Surface Database (ISD).

[cwhanse]

@cwhanse have look into this eeweather which supports Integrated Surface Database (ISD).

That's a neat package. I hadn't found that, thanks for posting it. I would have to use eeweather with pvlib for a bit, to figure out if there's value in new pvlib code as an interface.

Plusses:

• forked by volttron so eeweather could get long-term use.
• assists with finding stations by lat/lon

Minuses:

• long list of dependencies, pretty heavy for pvlib to import
• few developers but forked by volttron

Any response from siphon or metpy?

[nappaillav]

@cwhanse I will rise a issue in both and let you know soon siphon and metpy

[nappaillav]

I had raised a feature request in Siphon and metpy long ago, for which I got a response. The author feels that ISD reader function suits Siphon more than metpy.

But I'm not sure by when they will implement the function.

[cwhanse]

Seems like the responses are "PR welcome" and put get_ functions into metpy and read_ functions into siphon.

Having looked into it, I'm -1 on eeweather. Although that package has nice functions, it primarily serves a particular workflow, so maintaining a general interest function to read ISD is not likely a priority.

@wholmgren do you see a downside for pvlib users if the ISD functions live in siphon/metpy rather than native in pvlib?

[wholmgren]

No, I don't see a downside. You'd have to install another package or two, but their base requirements are not difficult to manage. @nappaillav I'm sure that the siphon/metpy maintainers would welcome a pull request.