Mapping data from Indonesia’s disaster information portal

Maps are great for decision-making (ex. where’s the nearest restaurant, how to get from point A to B)… they’re even better when you know how use them to help analyze data and information (thank you geography degree). A lot of data visualization automation software exists now that can produce charts, graphs and even maps to help see trends and patterns. But when it comes to really understanding and analyzing information, there’s still a lot to be said about including a human touch/perspective to data and information visualization.

One of the projects I’ve been working on is to capture and analyze disaster-induced displacement information for the Internal Displacement Monitoring Centre (IDMC) and it’s Global Report on Internal Displacement and Global Internal Displacement Database. One of things IDMC wants to know when a disaster strikes, like a flood, hurricane or earthquake, is how many people are displaced? It’s a simple research question that usually doesn’t lead to a straight-forward answer. Challenges can include lack of government monitoring for this kind of information, data collection and standardization issues, accessibility of said data, or even the political nature of publishing and sharing this information.


Fortunately some governments actually do a great job in collecting, processing, and publishing this kind of information. Indonesia is one of them. The government provides a disaster data portal which it maintains on a regular basis that tracks where a disaster takes place, when it happens, what kind of hazard triggered the disaster event, and the people killed, missing, injured and displaced/evacuated. For one of the most disaster-prone countries in the world, having this kind of information online, updated and easily accessible is an asset for research organizations like IDMC to be able to develop policies and recommendations that can have an impact on saving lives.

While the website has automatic visualization features, it requires a lot of assumptions and understanding by the user to know what to search for. At the same time, it is a bit challenging to use since it’s an online portal that has limited visualization and analysis capabilities. As part of my research, I decided to put my geography background to work to make sense of this data.

The mapping feature from BNPB disaster data portal -
The mapping feature from BNPB disaster data portal –

I downloaded the raw data in Excel format and in most situations a quick manipulation of Excel can reveal some trends. However the Excel included too many data points with differing variables like event date, hazard type, and location. I wanted to find a better way to make sense of its data so I decided to plot the data using QGIS, a free open-source Geographic Information System (GIS).

Here’s a quick summary of what I did:

  1. The Excel included raw district-level disaster information that goes back as far as 1815. I only need 2016 data so I filtered the data set and extracted all 2016 data that included “Mengungsi” or evacuation values greater than zero.
  2. In order to plot the data on a map, I needed to add spatial information to the data set. As the Indonesian data was broken down by districts, a quick search led me to district boundary level data published by the World Food Programme – unfortunately I couldn’t find district-level spatial data on the government website.
  3. Once I joined the Excel sheet with the district boundaries, I still needed to clean and verify that all districts in the government disaster data set matched the WFP district boundary data set. This is key otherwise the data can’t be mapped by QGIS.
  4. Since no GPS locations were included to pinpoint exactly where each disaster occurred, I defined a centroid (i.e. a point at the centre of each district boundary). This allowed me to plot each event as a specific point on the map to help in analyzing and aggregating information since multiple events can take place in one district.

It may not have been pretty, but it did make it easier to interpret the data based on hazard type, event date, and geographic location. And it made it more effective to work with when I wanted to conduct further analysis, run queries to address different research questions, and produce maps like the ones below.

Evacuations Events-by-Date Events-by-Hazard-Type Total-Events-by-District

Data visualization automation software and websites can be useful, but it’s also great to have a skill like old-school mapping and cartography to turn to when I need it… times and projects like these make me realize how useful a geography degree can be.

Cartographer takes Kobe to school

Admittedly the title for this post would make for a great story, but unfortunately it’s only fantasy for now. Skill and talent may still have a large part to play in basketball, but Kirk Goldsberry thinks there’s more to it and that thinking like a cartographer (i.e. you know, those guys that makes maps) might actually help people understand the game better and improve the way the NBA plays and manages it.

Goldsberry’s quest to map every moment of basketball really stood out for me in WIRED magazine last month. This excerpt from Mark Mclusky’s new book “Faster, Higher, Stronger” (Xmas present anyone?) is about maps and basketball. Two of my favorite things. I can’t help but think where was this research when I was studying Geography in university – I would’ve jumped at the chance to work on it.

Goldsberry’s research is different compared to that of data-analytics-driven baseball (i.e. Moneyball). He saw the constant flow of basketball as just a problem in information flow.

Unlike the static, state-to-state action in baseball, basketball is a constant flow. Players switch from offense to defense, from posting up to double-teaming. If a baseball player is a left fielder, you know the basic area he will patrol on defense. If a basketball player is a forward, he could be anywhere on the court at any time. The game has no states, so statistically you can’t determine the odds of a given outcome.

Basketball hoop

So the whole problem with basketball wasn’t so much percentages and probabilities, but of space… more specfically the spatial distribution of players in where they have their strengths when shooting, playing defense, or driving the lane.

Instead of focusing on the numbers that defined a state in baseball, Goldsberry began to focus on the locations and movement of objects—specifically, the players and the ball. It was a mapping problem… To understand basketball, you also have to understand space. You need a cartographer.

The best thing about this project isn’t so much the geek-factor of collecting stats and visualizing it, but what Goldsberry wanted it to do.

“I wanted to find a way to get this data to sing a new song, to tell us things like where Kobe is good and where Kobe is bad… and to communicate to players, and fans, and the media.”

By charting the location and frequency of every shot in the NBA, Kirk Goldsberry can create a map of the strengths and weaknesses of each player’s offensive game, like the ones below.

Midrange shots aren’t very productive for most players—except Nowitzki, who loves the right baseline.
Even the most prolific three-point shooter of all time has relatively weak areas, like from the left wing.

If this really is going to change the face of basketball like how Moneyball did for baseball, I’m looking out for a future movie. In the meantime, it would be great to see a head-to-head match up with Goldsberry and Kobe!

To see or not to see?

I can’t believe it’s been almost a month since I last wrote in this blog. Time sure does fly, especially when we head into the holiday season with Christmas just around the corner. Maybe one of my new year’s resolutions should be to write more often… It seems friends and family, as well as some other people, liked my recent article about how Crisis Mapping and new technologies can be used for reducing disaster risk – well at least that’s what Facebook is telling me. Anyway, it’s always nice to get positive feedback and encouragement, given that writing is such a personal thing and people are pretty free to tear things apart.

Are we (becoming) a visual society? The photos in this post taken during this year’s Fete des Lumieres in Lyon, France illustrate that we are (and have been) a very visual society. Given that we now live in a age where screens have become second nature, do we need (more) images and visuals to stimulate the way we feed on information? The CrisisMappers conference that sparked the idea for the above article I wrote provided a good insight into this issue.

Fete des Lumeries - Cathedral Saint Jean

From the surface, CrisisMappers ICCM2011, the 3rd INTERNATIONAL CONFERENCE ofCRISIS MAPPERS, seemed to be full of visuals and the way we can now collect and share information in real-time through our mobile phones, the internet, iPads, etc… But at the same time, I went to one of the side events where a group of 50+ people talked about how to better communicate the visuals – going beyond just dots on a map. So even with all this data and info that we can generate and collect, it seems we still have problems sifting through it all to tell a story. Do we need people who can both understand the technical bits of making things visual AND who can understand how all the visual and textual bits fit together to tell a story that makes sense to the world? Personally, I think it’s a good sign that we can’t just rely on making things “pretty”, and that we can’t just rely on the fact that the “facts/content speaks for itself”. It means that people working in the communication industry still have a lot of value – just as long as people are willing to evolve and change with the times, and understand the content and communication needs of the general public.

Fete des Lumieres - Place Louis Pradel leaves us wide-eyed

All this talk of “visuals” makes me think about people who are visually impaired or blind – how does our current technologies help them stay connected? But that’ll have to be another post!