The Most Interesting Mapper

The Most Interesting Mapper in the World

In case you live under a rock, this particular meme has its origins in a set of beer commercials that I find kind of amusing. Here is a decent compilation of them.

<sorry to interrupt this completely awesome post but quick note: This plugin has been turned into a core JOSM feature. See "Update 2" at the bottom of this post>

But seriously, the ImproveWayAccuracy plugin is a wonderful tool. Perhaps the prime use case to show it off is a smoothly curving road that was originally either mapped from low resolution imagery or (around here) imported from TIGER data that may have used far too few nodes to make a smooth curve. To fix this by hand is highly annoying. You have to drag nodes around and then grab the + between two existing nodes to create a new one and drag it to where you want it, repeating that as often as needed to achieve a smooth enough curve. Each node requires several precise mouse movements. 

It is much easier to draw a way from scratch where you just move your mouse along the curve and click as needed to form a smooth looking curve. Well, the ImproveWayAccuracy plugin makes this possible to do with existing nodes/ways!

Random computer upgrade notes

I recently did some reshuffling of the computers in my home. I built a brand new gaming system from parts I bought on newegg. Intel i5 Sandybridge, 16 GB of memory and an nvidia video card that not only has a 120mm fan, but has two of them. The insanity. Then I used the parts from the old gaming system (first generation i5, 12 GB, ATI graphics) to give my ancient linux box a major upgrade.

Of Linux and Distros
The Linux box was so old (AMD Athlon x2 4400+ with 3 GB) that it was becoming a pain to use for any kind of desktop applications. I ended up neglecting it and spending most of my time on my windows box (I know! The horror!) with multiple putty sessions open. Also, it was running Gentoo Linux. In my local Linux user group, we have determined that people who are over 30 don't use Gentoo. It is a worthy distro and has many strengths. I have been using it for over 10 years on some system or another and because of its minimal nature, I learned a lot about Linux that I would otherwise be oblivious to. But it can be a real pain, especially to maintain for desktop use. I was getting tired of every other emerge --sync generating package conflicts, lengthy config file merges and the occasional system toolchain breakage that you had to resolve before rebooting (or loosing power!)  otherwise the system would completely eat itself. So, being a distinguished 31 years of age now, I decided it was time to drop Gentoo. So long and thanks for all the fish.

Simple OpenStreetMap tile rendering with Maperitive

Lately there have been a lot of people popping into the OSM IRC channel (#osm on oftc.net) because they have been getting the "you've been bad" tile from the OSM tile server:

The reason is that they are using tile scraping applications that violate the OSM tile usage policy. Typically people just want to download tiles for their city or a vacation destination or travel route in order to pre-load them on a mobile device for offline use.

This is of course a perfect use case for OSM. The problem is that OSM is a project run by volunteers and with limited server capacity. Our primary product is geographic data, not map tiles. So when these tile scraping programs start hitting high zoom levels, it can cause serious problems for the tile server. The solution is to download OSM data instead of rendered tiles. Then you can render your own tiles or even use offline vector rendering.

One tool to make your own tiles is Maperitive. It is pretty easy to use and by default generates tiles that are fairly similar in style to the default mapnik style you see on osm.org although it is very customizable. It is a .NET application although it claims to work in Linux/OSX with mono. Also, while it is free to download, it is not open source which is kind of rare for the OSM world. These two things may cause some people to have an allergic reaction (I'm looking at you, Serge) but if you're still reading, here is a quick lesson on how to use it to render your own tiles. If you keep reading, you should be able to start rendering your own tiles in about 10 minutes. There is a tutorial video and some detailed documentation on the Maperitive website but I don't see a basic text based tutorial that walks you through the common "render some tiles for this area" use case.

Getting Started
Installation is pretty straightforward. Just head over to http://maperitive.net/ and find the "Download" link. Grab the latest .zip file (Maperitive-1228.zip right now) and download it. Once the download finishes, just unzip the Maperitive folder inside of the zip file somewhere and you're done. Open the Maperitive.exe file to fire it up.

Cleanup on the Map Aisle

Spills happen. A sticky honey jar in the grocery store. A glass of orange juice at breakfast. Oil spills in the Gulf of Mexico. Node spills in OpenStreetMap. Some can be cleaned up with a mop and some water. Others require a more technical solution. What is a "node spill" you ask? It's when nodes get uploaded to OSM that don't have any tags on them and aren't part of a way. These tagless, unconnected nodes add no useful information to the map and are just dead weight in the database. Where do they come from? There are at least a couple of common sources of node spills. One is editor bugs and simple user error. These are generally pretty small spills of 10s or maybe 100s of nodes. The bigger problem is imports.

The topic of imports deserves its own post. For now I'll just say that badly performed imports and insufficient checking after imports can lead to 10s of thousands of empty nodes. The basic problem is that nodes get uploaded first and don't become part of a way until the way is uploaded later. So if something goes wrong with the way upload, the nodes end up in an orphaned state.

I have come across a few of these nodes before but what really caught my attention was a failed NHD (National Hydrography Dataset) import in Oklahoma. It happened to poke up into Kansas a little bit where I noticed it next to a state highway I was editing. After a lengthy thread on the talk-us mailing list I eventually found a good way to detect these nodes. Since then I have made my way across the US and some of Canada deleting these useless nodes from the database. As detailed in this message it goes something like this:

• Use the XAPI to perform a query of the form /node[not(way)][bbox=a,b,c,d]
• Open the result of this query in JOSM and apply a filter to hide all nodes with tags
• Perform some checks to make sure there really is no useful data
• Delete nodes and upload

The Main Attraction

I have mentioned the TIGER import several times. The initial import tagged roads with abbreviations in their name. For example "Ave" instead of "Avenue" or "St" instead of "Street." However the general rule in OSM is to not abbreviate things. So a script was devised to automatically expand abbreviations. This script was run by the user balrog-kun. Until tonight I assumed that it had been run over all of the TIGER data. But then I saw a few abbreviations while working on some highways in Kansas and decided to dig a little deeper. I did some XAPI queries (using the database I talked about in a previous post!)  to see if anything interesting stood out. I did one query for all streets named "Main St" and another for all roads named "Main Street" and here is what I got back.

Green is "Main St" and the rest is "Main Street"

One Mapper, One Year

Today is the one year anniversary of my first OpenStreetMap edit! I thought I would take the opportunity to reflect on how OSM and myself have had an effect on each other. My intention is not to brag about all the mapping I have done but rather to point out some different ways in which contributions can be made and some of the tools I use when mapping.

My very first edit was to add a foot/bike path near my apartment so that it would show up on OpenCycleMap. Unfortunately the imported TIGER roads were so far off that I had to realign them before I could add the path in the correct location. And of course correcting the ones right around the path led me to correct the ones connecting to those and the ones connecting to those and so on... At the time I didn't even know what TIGER was or what all those tiger:* tags were on all the roads.

Some highlights since then:

• GPS trackpoints uploaded: 283,271 in 75 traces (most from my Garmin Edge 305)
• 410 changesets created using Potlatch, JOSM, OsmAnd and Vespucci
• Pictures taken: over 2,000 (4.5 GB saved)  Most were geotagged to a GPS trace to aid in mapping features (uploaded some of them to openstreetview.org)
• Some of the locations I have edited: 
• My university campus
• My city
• My state
• My country
• My high school
• My childhood home
• Some random place where there was an earthquake
• Map objects affected (according to hdyc.neis-one.org - last updated in January)
• Nodes: about 165,000
• Ways: about 24,500
• Relations: about 500 (I created over 200 county relations in 3 states since the last update)
• Contacted multiple government agencies in Kansas to get explicit permission to use their data to assist in mapping. Documented these data sources on the Kansas page in the wiki
• Started a blog (self referring link!)

Now for a pretty picture. Here are all the GPS traces in my city. I think there may be one or two from other people but most of it is me.

Keeping a database up to date with osmosis

This is a follow-up to my previous post about importing a planet file into a pgsnapshot database schema. However the same basic process applies to other schemas that osmosis can write to.

To briefly set the stage: After downloading the weekly planet file and processing/importing it into a database, the data is now probably 3 or 4 days old. OpenStreetMap provides both minutely and hourly change files that can be applied to the database to bring it up to date with what is live right now in the master OSM database. If you think about it, that is actually a pretty sweet deal. Someone on the other side of the planet can spot a new restaurant while on their lunch break, use an application on their phone to upload it to OSM and you will get the change applied to your local database in less than  120 seconds. Does this blow anyone else's mind just a little? Open data is awesome like that.