[Wiesbaden] Data

[jklmnn]

Wiesbaden also has parking data available at https://www.wiesbaden.de/leben-in-wiesbaden/verkehr/verkehrsinfos/parken.php.
The raw data seems to be at https://wi.memo-rheinmain.de/wiesbaden/parkliste.phtml?order=carparks. While it looks quite ugly to parse it seems to have all the data we need.

[kiliankoe]

While it looks quite ugly to parse

Lol, this is far better than most others 😄

[jklmnn]

But only with whitespace correction 😄

[AugustQu]

Hi,

here I am.
From what I understand: first I deliver the coordinates of the parking-lots given in the URL above.

So this may look like:
static_data["City I"] = {'latitude': "50.08121", 'longitude': "8.23637", 'adresse': "Schwalbacher Str. 41-43"}

Am I correct?

And a question:
this parking-lot is closed on sundays/holidays and opened on monday to saturday from 7:00 to 20:00. How do I enter this information?

PS: I've taken this file as an example:
https://github.com/Bytespeicher/pyParkingLotEF2Json/blob/master/pyParkingLotEF2Json.py

[AugustQu]

I've seen a lot of py-files. Looking at the code I found:

• create a loop over all parking-lots (14 for WI)
• search for the name (td-Element)
• next is the 'Freie Stellplätze'
• finally: return the name together wit the numbers.

Am I right?

PS: I'm new to python.

[jklmnn]

Hi,

here I am.
From what I understand: first I deliver the coordinates of the parking-lots given in the URL above.

So this may look like:
static_data["City I"] = {'latitude': "50.08121", 'longitude': "8.23637", 'adresse': "Schwalbacher Str. 41-43"}

Am I correct?

And a question:
this parking-lot is closed on sundays/holidays and opened on monday to saturday from 7:00 to 20:00. How do I enter this information?

PS: I've taken this file as an example:
https://github.com/Bytespeicher/pyParkingLotEF2Json/blob/master/pyParkingLotEF2Json.py

The project you are referring to seems to be outdated. Also it doesn't use our API.

The files relevant for you reside in park_api/cities. I'd recommend you to take a look at Sample_City.py and Sample_City.geojson. You could copy those files to Wiesbaden.py and Wiesbaden.geojson respectively and then edit them. Wiesbaden should then automatically appear in your local instance.

For parsing we use beautifulsoup4. I'd advise you to do the same since it is easily powerful enough for the task and already a dependency. About the data the relevant table is accessible via the xpath /html/body/table/tbody/tr[3]/td/div/table/tbody and then iterate over all trs except the first one (which is the table header) and get the data from the tds.

Another far simpler, but important and time consuming task would be filling the geojson file which needs to be done by hand but only once. The most important information are the coordinates.

[AugustQu]

Hi,

I'm not only new to Python but also new to Github.

I copied the whole project as a ZIP-file, expanded it and modified the file Sample_city.geojson adding all the 14 parking-lots of Wiesbaden. Finally I saved it as Wiesbaden.geojson, locally on my PC.

Some questions:
#1: in the 'features' of this file coordinates have to be entered. I entered the coordinates of the city town hall. Is this correct?
#2: in the 'properties' I have to enter an URL. Here I entered "https://www.wiesbaden.de/leben-in-wiesbaden/verkehr/verkehrsinfos/parken.php". Correct?
Also as a source I entered: "https://wi.memo-rheinmain.de/wiesbaden/parkliste.phtml?order=carparks". Correct?
#3: there is one parking lot that has 2 entries for cars. Can this be handled? Or shall I omit the second entry?

[jklmnn]

I'd recommend to check out the GitHub Hello World tutorial and the git tutorial (while GitHub builds upon git, git is a free version control system). There also is a collection of resources that looks like a good start.

As for the general workflow, you should fork this repository, do your changes on your on fork and the create a pull request with your changes for us to review.

1. The data includes a list of features with the first being the city itself and all others being parking lots. The coordinates of the first feature should be either the geographical center of the city or the geographical center of the distribution of parking lots. We usually take the coordinates of large map providers for this.
2. Yes that's both correct. The url field denotes the public accessible location of the data (city homepage, etc) and the source denotes the actual source of the information that the parser will receive.
3. Do you mean entries in how you enter the lot or data entries? In the first case we only use one address and coordinate. In the second case I'm not sure how it would look like.

[AugustQu]

Hi,

thanks for the link to 'hello world' on GitHub. I will look at it.

Now I have the file Wiesbaden.geojson ready I will check it again, polish it a bit and then open a Pull Request.

In parallel I will try to work on the file Wiesbaden.py.

[jklmnn]

Thanks! I added a branch wiesbaden so when you open the pull request please select this as the target branch since we only allow finished features on master. When everything works wiesbaden will then be merged into master.

[AugustQu]

playing with Python I got this result:
my code:

table=soup.select('table')
td = table[2].find_all("td")
i=0
while i < len(td):
print('name:',td[i+1].text.strip())
print('frei=', int(td[i+2].text.split()[0]))
print('total=', int(td[i+2].text.split()[2]))
i += 5

the result, first entry only:

('name:', u'City I')
('frei=', 137)
('total=', 165)

hint: I added the html statically for my test.

So I have to add these informations to the data-object and I should be done.

Looking into the Sample_City.py I find these lines:

    data["lots"].append({

        "name": lot.name,
        "free": lot_free,
        "total": lot_total,
        "address": lot.address,
        "coords": lot.coords,
        "state": state,
        "lot_type": lot.type,
        "id": lot.id,
        "forecast": False,

The first 3 lines are the information given above, the rest comes from the geojson-file looked into via the name.
Correct?

PS: I have to check for state and lot.id.

[AugustQu]

I added this to my code:

stand=soup.select('span')
last_updated_date=stand[0].text.strip().split()[1]
last_updated_time=stand[0].text.strip().split()[2]
print("last updated=", last_updated_date, " ", last_updated_time)

which results in this line:
('last updated=', u'07.04.2019', ' ', u'11:16')

[AugustQu]

I just added the files Wiesbaden.geojson and Wiesbaden.py but I'm not sure that I did it correctly.

In the Python-script I added 4 TODOs which describe lines to look for:

• the handling of last_updated
• the state (twice)
• the lot.id which I do not know

Can you please check these lines and also the whole script.
Thanks.

[AugustQu]

one error I made:
I loaded the file Sample_city.geojson, replaced the contents with my file (Wiesbaden.geojson) and saved it as Wiesbaden.geojson.
Instead I should have created a new file via the button.

Same for the py-file.

Sorry.

[jklmnn]

Sorry for my late reply.

The first 3 lines are the information given above, the rest comes from the geojson-file looked into via the name.
Correct?

Yes that's true.

There are some issues about the pull requests. One is they're opened against master which we won't merge until everything works perfectly. But you can easily choose the wiesbaden branch with the dropdown on the left:

The second thing is I'd suggest you only make a single pull request with all your changes. This helps us to directly comment on all parts of your code to help you better. By pushing to the branch of the single pull request it will be updated and we always will see the changes.
Furthermore having all changes on a single branch helps us checking it out and testing it more easily.

[AugustQu]

what do I have to do now? Add a new Pull-request against the Wiesbaden-branch?

Or do you handle this situation?

[jklmnn]

Yes, you have to open a new pull request against the wiesbaden branch.

[AugustQu]

OK. I added the files Wiesbaden.py and Wiesbaden.geojson in the cities-directory of the Wiesbaden-branch and did a Pull-request. I hope this is correct now.

Unfortunately I added the file Wiesbaden.py to the wrong directory in the first try so please delete this file. And also it looks like I did overwrite the file Sample_city.py with the code for Wiesbaden. Can I correct this? Or do you simply reject the 'modifications'?

[jklmnn]

Thanks, that looks better. Since the overwritten Sample_City.py file is in its own commit it doesn't pose a problem as I can omit it when merging.
The current process will look as follows:
I'll review your code, probably request some changes which you then can push to the pull request branch. When we're at the point that the code is clean and working we'll merge the pull request.

[AugustQu]

I will look at the status of the parking-lot over the weekend, then make some changes to the code.

I've seen that the data on wiesbaden.de showed the status of 2 parking-lots as closed, I saw this last night. I will try to find out how they do it.

[AugustQu]

it looks like there are only minor modifications needed. So I will look for the status, make the needed changes plus the already mentioned modifications and deliver them together.

I gues the next version will come on monday.

One hint: the word Uhr is included in the following line taken from the file Sample_city.py:

    "last_updated": convert_date(last_updated, "%d.%m.%Y %H:%M Uhr"),

[jklmnn]

One hint: the word Uhr is included in the following line taken from the file Sample_city.py

This is of course just an example and needs to be adapted to the actual data.

[AugustQu]

OK, I got it.

Uploading my code to the branch wiesbaden I did encounter 2 problems:

• I do not see the old files any more

• so I tried to create new files but could not save them because I have no write access

What can I do now?

[AugustQu]

there is this: AugustQu/ParkAPI
there is a wiesbaden-branch where I can find my files.

I will add my latest modifications to these files and then make another pull-request.

[AugustQu]

so I hope I did it correctly.

[AugustQu]

I made some modifications to the py-script and will upload this file later. These modifications were of the type beautifying.

But I do see one problem: there is a parking-lot with the name Coulinstraße. On the memo-rheinmain-page it appears as: Coulinstraße, in the geojson-file I added it as Coulinstrasse. In the py-script this will be handled as:
parking_name = td[i+1].text.strip() lot = geodata.lot(parking_name)
so the name is extracted from the memo-rheinmain-page and this is used for the search in the geojson-file. This will result in no match.

How can I handle this? Enter the name in the geojson-file as Coulinstraße?

[AugustQu]

I found a typing-error:
in the geojson-file the name of a parking-lot is written as: Rhein Main
in the memo-rheinmain-site it is written as: RheinMain

I will fix this.

Put this aside: does the script work?

[jklmnn]

Sorry for my late reply, I'm quite busy at the moment.

so I hope I did it correctly.
It seems you pushed your changes to the already existing pull request, so yes that's correct. I will review it when I find time.

About ß in street names, they should be usually encoded in UTF-8 that supports those characters. Although if the page doesn't have the correct encoding they will be erroneous. I think we have a solution for this to at least convert those characters to ASCII (@kiliankoe do you remember how we fixed that in other cities?).

Put this aside: does the script work?

Same as above, I can tell when I find time to review (and test ofc.).

[AugustQu]

Sorry for my late reply, I'm quite busy at the moment.

OK. The usual thing: I created some lines and want to see the result immediately.....

I've jut added some minor modifications to the two files. One ist the name 'Coulinstraße' in the geojson-file and the other is a bit of cleaning the code in the py-script.

[jklmnn]

Thanks for the changes, I reviewed them. There are two stray lines in the geojson that shouldn't be there. If you remove them I think it's ready to be merged.

One last thing. There is the active_support flag in the json which decides if the city is enabled by default in the apps. If you want the city to be enabled by default I'd suggest you set it to true.

If the website changes and the parser breaks (happens usually rarely) we will either try to fix it or (if the fix requires more work) we probably set active_support to false again unless someone fixes it. That what the flag is for.

[AugustQu]

I've seen you updated your master branch but not your wiesbaden branch.

I looked into my branch and do not see a file Wiesbaden.py in my master branch, but I do see it in my wiesbaden-branch.

And the pull-request is made from AugustQu/wiesbaden against offenesDresden/wiesbaden.

Can you please check this again.

[AugustQu]

regarding whitespace:

my editor here does not show this. And is your system so sensible?

[AugustQu]

whitespace:
in my file here there is no whitespace at the end of that line. So (again) it looks like a copy-paste-problem.

[jklmnn]

Trailing whitespace doesn't necessarily break anything on the running server. But it is at least unclean code which we want to prevent as much as possible and it may break data that spans over multiple lines if ignored.
Related: https://softwareengineering.stackexchange.com/a/121560

Also it generates changes in files that are no longer easily visible creating different versions of the same software that have no visible change. Example:

In this case a trailing whitespace has been introduced but for the reader it is not clear what has changed.
Also there are editor(-plugins) that show trailing whitespace. When I open this file in Vim the whitespace is marked in red:

[AugustQu]

But it is at least unclean code

OK.

I have used gedit for editing the script but I can switch to eclipse which shows whitespace.

Nevertheless this whitespace was entered by copying the lines from my file to the file here.....

[AugustQu]

3 simple questions:

#1: in the app I've seen lines in green, yellow, red and blue.
green lines mean a lot of free space - OK.
yellow lines mean percentage has fallen under a value of ?? (which value is it?)
red lines mean only some parking spots are available (from which percentage on the color will change?)
blue line means ?????

#2: I've seen texts like 'no data availble'. Will this text appear when my code sets parking_state = 'nodata' ?

#3: closed parking-lots disappear from the list. Is this the intended behaviour?

[jklmnn]

I assume you talk about the Android app which I wrote. I can't tell you about the iOS and Windows apps.

1. green: 100% free - 20% free
   yellow: 20% free - 5% free
   red: <5% free, closed
   blue: no data available
2. yes
3. If you tap the three dots in the upper right corner and select settings you get to the settings dialog. There you can choose which parking lots are shown. By default full (0% free) and closed lots are hidden.

[jklmnn]

I checked the map and it seems to work now, thanks for the fix.

[AugustQu]

YES! Looks good.

I will write a text in my blog about this app but this will take some time......

[AugustQu]

I assume you talk about the Android app

Yes. And thanks for the explanations.

[AugustQu]

another question:

in the top-line of the app there is a world-icon. Tipping on it shows some circles in green/red.

What is this good for?

Should it show the location of the parking-lots via OpenStreetMap? On my Smartphone I only see a background in grey.

[jklmnn]

The OpenStreetMap library the app uses requires the permission to store/access local data to store its tiles. Depending on your Android version the app should have asked for that permission. If not you might enable this permission for the app in your system settings. If this permission is not given the tiles sometimes don't show up.

The apps source code is available at https://github.com/jklmnn/ParkenDD. So you can take a look at it if you want to (and know how Android apps are built). Unfortunately I don't have the time to maintain the app anymore (I only fix breaking bugs that lead to crashes/incompatibilities with newer Android versions).

[AugustQu]

here is the announcement of the availability of Wiesbaden-data on my blog:
https://linux-fuer-wi.blogspot.com/2019/05/parkendd.html

[jklmnn]

The article looks nice. I have only one issue. The city that is shown after starting the (Android) app is the closest city if location data is available or Dresden if location data isn't available. So for you if you have a location Wiesbaden should be selected by default.

[jklmnn]

I finally found some time to write a short tweet: https://twitter.com/parken_dd/status/1127983797667020801

[AugustQu]

Thanks.

[jklmnn]

It seems Wiesbaden has moved it's parking information to https://www.wiesbaden.de/leben-in-wiesbaden/verkehr/auto/parkhaeuser.php. The map consists of different parking lots, each linking to their own site with the relevant information. I don't have an idea how we would use this at the moment. I'll reopen this issue and disable Wiesbaden until we find a solution since the old source doesn't seem to be available anymore.

[AugustQu]

OK. I will look at it.

[AugustQu]

somehow I found this link: https://mobileapps.ffh.de/iphone/parkhaeuser/parkhaeuser.json?city=wiesbaden

There are 2 problems with it:
#1: can we use it?
#2: there are differences in the number: the wiesbaden-site reports 170 free parking lots for one location, the link reports 188 for the same location.

it looks like sites like:
https://www.ffh.de/verkehr/parkhaeuser/parkhaus-info-wiesbaden.html
https://www.harmonyfm.de/service/parkhaeuser/wiesbaden.html
https://www.planetradio.de/service/parkhaeuser/wiesbaden.html

use the same data-source.

[jklmnn]

The data source looks good on a quick view. After all it's JSON so much better than a website. The data difference may be caused by their update intervals (e.g. the data could be 5-10 minutes old which is reasonable for such a difference). We should check that, maybe there is a timestamp to compare somewhere.
So I think we should use that as it currently seems to be the best option.
Thanks for looking it up!

[AugustQu]

there is a line in the JSON-file saying:

statusTime | "21.05.2021, 15:51 Uhr"

To me the data looks like very old. I will check this over the day.

And I will try to contact someone in Wiesbaden over the weekend.

[AugustQu]

ich habe dazu mal etwas geschrieben:
https://linux-fuer-wi.blogspot.com/2021/06/open-data-worte-und-taten.html

[AugustQu]

es sind etliche Monate vergangen, aber am Status hat sich nichts geändert. Hier ist meine Beschreibung samt Kommentar dazu:
https://linux-fuer-wi.blogspot.com/2021/10/parkendd-weiterhin-ohne-wiesbaden.html

Ich will dran bleiben. Aber, wie sagt man bei uns im Dorf: da kannste ach em Oxx ins Horn petze.

[jklmnn]

From what I saw at least it looks like they aim to provide a working solution in the near future (early next year). Thanks for tracking this!

[AugustQu]

Yes, someday they will offer a new solution. Maybe.

Btw: currently they have a lot of problems, keyword Salzbachtalbrücke.

[AugustQu]

maybe things will change (a big maybe).

Please look at this document: https://piwi.wiesbaden.de/dokument/1/2818223

Coming tuesday this event will take place: https://piwi.wiesbaden.de/sitzung/detail/2718746/tagesordnung/oeffentlich

We will see whats happening then. I will be there and listen to the discussion.

[kiliankoe]

Thanks for being so involved!