Filtering map markers with Leaflet.js: a brief technical overview

Project Description:

ParrotNet is a network of academics, professionals and policy makers that are investigating various characteristics and effects of non-native parrot species. The sub-section of this project I’ll be referring to in this blog post is a map that’s part of the virtual European monitoring centre which lives on the University of Kent website. This map exists to display data records of invasive species of parrots and their sightings. The intention being to represent data from Ebird and GBIF ( potentially other sources as well) in a visual manner, with the ability to filter records based on species, number of observations, location, data source and the year of the sighting.

Those were the key initial requirements and to start with the data is for rose-ringed parakeet observations only, with the intention of adding more species as time goes on. This is what our final map looks like:

ParrotNet map

Technical Aspects

Having worked with Leaflet on other projects such as EWTO, we already had a good pool of knowledge and the infrastructure ready for implementing custom maps, including the central maps server used by /maps. In the past we’ve used Tilemill for creating custom map tiles combined with datasets from Natural Earth. We love maps and with so many map-related projects completed and more on their way, we thought it was about time to share some of the knowledge we’ve gained. This is the first in our series of technical blog posts, so any feedback is greatly appreciated.

Now the introduction’s out of the way let’s get our hands dirty! (From here on out if you’re not a techie I may as well be talking gibberish)

Extending Leaflet with custom plugins

Like most modern JavaScript libraries, Leaflet is easy to build upon, you just extend the core classes and override methods where necessary. A few custom plugins were created for this project, their structure is reflected below.

Diagram of Leaflet plugins

This diagram illustrates the class hierarchy.  The following table quickly summarises what they do and how they interact with one another:

Class Description
CustomControl Allows adding leaflet controls anywhere within a page, not just within a map in the designated control corners.
Filters Handles the top menu bar and setting up filters within it.
FilterDropdown Displays a filter dropdown with items that can be “multiselect”, “morethan” (1+, 2+, 5+ etc) or “single select”.
FilterSlider Responsible for the year slider, the slider tooltip and any filter logic behind it.
Handle This is the handle that is dragged across the slider, it is used to calculate the current position on the slider and perform any snapping calculations.
FullScreen A simple fullscreen toggle button.
Infobar Displays the bar at the bottom of the map which shows and formats what filters are selected and allows you to deselect them all at once.

To illustrate how simple it can be to extend core classes, we’ll roughly recreate our CustomControl class.

L.Control.CustomControl = L.Control.extend({
	setPosition: function(position) {
		var controls = this._controls;

		if(controls) {
			this.remove();
		}

		this.options.position = position;

		if(controls) {
			controls.addCustomControl(this);
		}

		return this;
	},

	addTo: function(map, outercontainer) {
		this.remove();
		this._map = map;
		this._container = this.onAdd(map, outercontainer);

		L.DomUtil.addClass(this._container, 'leaflet-custom-control');

		var child;

		if(this.options.position && (child = outercontainer.childNodes[this.options.position])) {
			outercontainer.insertBefore(this._container, child);
		} else {
			outercontainer.appendChild(this._container);
		}

		return this;
	},

	remove: function() {
		if(!this._map) {
			return this;
		}

		L.DomUtil.remove(this._container);

		if(this.onRemove) {
			this.onRemove(this._map);
		}

		this._map = null;

		return this;
	}
});

L.Map.include({
	addCustomControl: function(control, addTo) {
		control.addTo(this, addTo || this._controlContainer);
		return this;
	}
});

As you can see, it’s straightforward enough. This gives us the optional “outercontainer” element to insert our control within and the position it should appear within this container, rather than say “topleft”. To add a custom control to our map within a fictitious element we would do this:

map.addCustomControl(new L.Control.CustomControl(), L.DomUtil.get('id-of-element'));

Applying multiple filters to Leaflet markers

If we wanted to filter markers based on a single attribute then we could achieve this easily with LayerGroups. We would add markers to layers that correspond to their attribute, so all markers with the attribute “a” would be on the “a” layer and we would simply toggle layer “a” on or off. This doesn’t fit our use case as we need several different filters to be applied at once (show markers with attribute “a”, “b”, “f” and “z”). No pre-existing solution exists for the kind of filtering we’re talking about and the simplest possible solution is to perform a linear search which is O(n). This means the more markers we have the slower the search algorithm will perform. We considered this a good compromise for our needs, although there are ways of speeding the process up significantly, as mentioned at the end of this post.

The basic idea is to parse our initial data and store the markers in an array. When a filter is applied we loop through our array of markers checking if they match the selected filters and, if they do, add them to the map. This is a rough idea of how it would work:

/*
map              = our Leaflet map
allmarkers       = array containing all our markers
displayedmarkers = layer containing the markers currently shown
appliedfilters   = array storing the currently selected filters
data             = data to check against
*/

function applyFilters() {
 	displayedmarkers.clearLayers();
	for(var i = 0; i < allmarkers.length; i++) {
		if(matchesFilters(allmarkers[i])) {
			displayedmarkers.addLayer(allmarkers[i]);
		}
	}
}

function matchesFilters(marker) {
	for(var i = 0; i < appliedfilters.length; i++) {
		if(!marker.options.data[appliedfilters[i]] ||
			marker.options.data[appliedfilters[i]] !== data[appliedfilters[i]]) {
			return false;
		}
	}
	return true;
}

Lessons learned and future improvements

Debounce CPU intensive tasks when necessary

Debouncing is effective when we’re triggering some piece of logic over and over again unnecessarily and it’s resource intensive. We basically delay running a function until a certain time has passed since the last time it was invoked. A common use case is for the “window.resize” event, say we have a function attached to this event that recalculates the layout of a page when it is resized. Every tiny pixel change to the window size fires the event which invokes our function, this is some heavy stuff, so debouncing would help alleviate costly operations running non-stop. Debouncing was useful for us when our year slider was moved and triggered a task that searches through all our markers. If a user went from 1967 to 2014, no matter how fast, for every year between those dates a search through all of our markers was triggered. We wanted to avoid this. We set a timer that waits 10ms and if the our search function hasn’t been run in that time then we run it. Even a tiny 10ms wait like this (barely noticeable) makes things snappier if people slide quickly through the years.

Use events in Leaflet to simplify code logic

Something that became apparent during development is that passing data to and from different components can become cumbersome. For example, the main FilterMap class would pass down what function to run when the slider was moved, and this went through the filters class, then to the filterSlider class. When a filter was changed it would run that function, also telling the infobar what had changed by passing that infomration back down the chain of classes… Before you know it your code’s a mess and you can’t even decipher what it is you were originally trying to do! The built in events system in leaflet can make this whole process a breeze. A filter changes so we fire a “filterchange” event, passing along any relevant data… meanwhile the main FilterMap and the InfoBar classes simply listen for this event and act accordingly. Events help promote the idea of separation of concerns and that’s something we should all aspire to… amen.

The Leaflet documentation is sparse

For your average map the Leaflet documentation is perfectly adequate, however when your usage becomes more complicated, requiring custom plugins, it’s necessary to look through the code to understand how things work. This is just something minor and luckily for us the code is clean and easy to understand!

User interaction isn’t always predictable

This is somewhat obvious, but it’s worth mentioning that users, especially those who aren’t necessarily familiar with the subject at hand, may find navigating a map with lots of filters etc. a little difficult. During development we tried to make things as usable as possible by ensuring what was happening was obvious and employing well established UI patterns. An example would be using radio buttons to signify that only one option may be selected while checkboxes mean several may be selected. We also included the information bar at the bottom that constantly displays the enabled filters. Another idea we had was to introduce a short interactive walk-through of the map, including how to use the filters and what the data signified. A possibility for the future is to use something like intro.js, which can be easily integrated into most projects with minimal effort.

Better data structures for efficient filtering

There are alternatives which could make the process of searching through large amounts of data faster. As mentioned previously we chose to run a linear search due to the fact our dataset is comparatively small. The search algorithm is based on the fact that our data is stored within each marker, in other words, the markers have attributes associated with them. A more astute way of doing this, and one that’s well known by the major search engines, is to reverse this relationship and associate the attributes with their markers (or pointers to our markers such as their index value within an array). This is what’s known as an inverted index. The concept is simple, but can afford us some great speed increases. The lookup time to find the list of markers matching each selected filter would be O(1) per filter, and then it would simply be a case of finding the markers that are within those lists eg. A ∩ B  ∩ C  ∩ D (A, B, C, and D being the sets of markers matching filters). Overall inverted indexes would greatly reduce the amount of markers to search through, although the benefits at this scale are minimal.

Finally here is a link to our map. *Holds twitchy finger away from keyboard to stop writing more*… that was much longer than planned! Keep your eyes open for another post next week. Hopefully you’ve been able to glean some useful information from this blog post and we look forward to writing about more projects in the future.

One response to “Filtering map markers with Leaflet.js: a brief technical overview

Leave a Reply