From 4fc7215a02bc7f26fc9b56e781b319bdb7115f27 Mon Sep 17 00:00:00 2001 From: John Wiseman Date: Mon, 3 Feb 2020 16:37:29 -0800 Subject: [PATCH] Updated README. --- README.md | 142 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 85 insertions(+), 57 deletions(-) diff --git a/README.md b/README.md index a9ebd81..c40bf92 100644 --- a/README.md +++ b/README.md @@ -1,19 +1,55 @@ # Advisory Circular -This is a twitter bot that alerts you when aircraft fly in circles. -You can see it in action at -[@SkyCirclesLA](https://twitter.com/SkyCirclesLA). +This is a twitter bot that tweets when it detects aircraft flying in +circles. + +The bot uses data from the [ADS-B +Exchange](https://adsbexchange.com/), which is a network of amateurs +who track aircraft, to monitor all the aircraft flying in a particular +geographic region. When it detects a +[helicopter](https://twitter.com/SkyCirclesLA/status/1224376709048193024), +[fixed wing +aircraft](https://twitter.com/SkyCirclesDC/status/1223995896536240133), +[blimp](https://twitter.com/SkyCirclesLA/status/1213310909302493184), +or +[autogyro](https://twitter.com/SkyCirclesLA/status/1216464781869121536) +flying in circles, it will tweet a short description of the aircraft +and an image of the aircraft's track on a map. ![Example tweet](Screen Shot 2019-12-13.png?raw=true "Example tweet") +The text of the tweet includes the aircraft's registration (if known, +otherwise just its ICAO code), type, speed, altitude, and transponder +squawk code. It will also include the city over which the aircraft is +circling, and a nearby landmark—a park, school, restaurant, or some +other notable place. + +The bot takes a screenshot of the aircraft as it currently appears on +the ADS-B Exchange's "Global Radar View" web page. + +This code currently powers the following bots: + + * [@SkyCirclesLA](https://twitter.com/SkyCirclesLA), which covers + the Los Angeles metro area. + * [@SkyCirclesSF](https://twitter.com/SkyCirclesSF), the San + Francisco Bay area. + * [@SkyCirclesDC](https://twitter.com/SkyCirclesDC), for the + Washington D.C. area. + +## Technologies used + This bot is written in Clojurescript, targeting Node. -The bot uses the adsbexchange.com API to get aircraft data in -realtime. It uses the Pelias front-end to OpenStreetMap and -WhosOnFirst data to come up with some color description of the -aircraft's location. It uses an embedded Chromium browser to take a -screenshot of the aircraft on the adsbexchange.com "global radar -view". +The bot uses the [ADS-B Exchange +API](https://www.adsbexchange.com/data/) to get aircraft data in real +time. + +It uses the [Pelias](https://pelias.io/) front-end to OpenStreetMap +and WhosOnFirst data to come up with some color description of the +aircraft's location. + +It uses an embedded Chromium browser to take a screenshot of the +aircraft on the ADS-B Exchange "global Radar View". ## Criteria for tweeting @@ -38,6 +74,9 @@ The bot will consider an aircraft to be "circling" if: about. It won't be eligible to be considered circling/tweeted about again until it stops circling and 30 minutes pass. +Many of the specific numbers in the above criteria can be customized +on a per-bot basis. + There are a lot of weird flight paths that can meet criterion #1, and I kind of like it that way. For example, see https://twitter.com/lemonodor/status/1203033917483175936 (consider @@ -48,24 +87,48 @@ https://twitter.com/lemonodor/status/1204550831103590400). ## Prerequisites to running the bot **Set up a Pelias instance containing information for the geographical -area you're interested in.** See https://github.com/pelias/docker and -if you're lucky the `projects` folder will have a predefined project -that covers the area you want. I run one using the `los-angeles-metro` -project. +area you're interested in.** Pelias is the geographic database that +the bot uses to know which city aircraft are circling over and find +nearby landmarks. It's also used to filter out aircraft that are +circling over airports. See https://github.com/pelias/docker and if +you're lucky the `projects` folder will have a predefined project that +covers the area you want. I run one using the `north-america` project. **Get an API key for adsbexchange.com.** See -https://www.adsbexchange.com/data/ If you have a receiver and feed +https://www.adsbexchange.com/data/. If you have a receiver and feed them data, they'll give you a key for free. Alternately, find another data source of realtime (or not) aircraft data and massage it into a form that the bot can work with. -**Get access keys for a twitter account.** Follow the [instructions at +**Create a twitter account and get app access keys for it.** Follow the [instructions at botwiki](https://botwiki.org/tutorials/how-to-create-a-twitter-app/). +**Get a `basestation.sqb` file.** `basestation.sqb` is a crowdsourced +sqlite database file that contains registration and type information +on aircraft. There is no canonical or official version of this file, +unfortunately. Honestly, it's a real mess that somehow many people in +the planetracking hobby are convinced is a good thing, or at least not +a problem. In any case, you can try downloading a version from +http://www.kinetic.co.uk/resourcesdownloads.php#4. + ## Running the bot -Put your adsbexchange.com API key and twitter keys in `secrets.yaml`. +Put your ADS-B Exchange API key and twitter keys in `secrets.yaml`. + +Create a `config.yaml` that looks like this: +```yaml +adsbx: + url: https://adsbexchange.com/api/aircraft/json +pelias: + url: http://:4000/v1 +lat: 34.1335 +lon: -118.1924 +radius-km: 10 +basestation-sqb: clean-basestation.sqb +``` + + Create an initial history database: @@ -83,50 +146,15 @@ Compile the clojurescript: Run the bot: ``` -node out/script.js --adsbx-url https://adsbexchange.com/api/aircraft/json \ - --pelias-url http://my-pelias.local:4000/v1 \ - --lat --lon --radius ` - -Specify your geographical region of interest by specifying a circle at -``, `` with a radius of `` miles. - -The bot will run once. The adsbexchange.com API only provides -instantaneous positions, so the bot will have to run many times and -collect some history before it's able to detect any circles. - -``` -while : -do - node out/script.js --adsbx-url https://adsbexchange.com/api/aircraft/json --pelias-url http://my-pelias.local:4000/v1 --lat --lon --radius - sleep 10 -done +while :; do python intervalexec.py 10 node out/script.js; sleep 5; done ``` +`intervalexec.py` is a script that will try to run the bot script no +more than once every 10 seconds. + + * **Do not run the script with a large radius, or adsbexchange.com will - ban your IP.** I use a radius of 40 miles. + ban your IP.** * **Do not run the script more ofen than once every 10 seconds, or adsbexchange.com will ban your IP.** - - -## To do - -- [ ] If we don't have neighbourhood or locality (just county), try to - use the locality of a nearby venue. E.g. "2 miles from - Glendale." -- [ ] Solve the issue of needing a differently sized buffer zone for - different airports--2 miles distance from LAX is still - practically inside the aiport. Add list with custom airport - coords and radii? Get access to OSM polygons and compute our own - distances? -- [ ] Add type information to tweet status, e.g. "Eurocopter AS350". -- [ ] Lookup registration info. Possible sources: FAA, - adsbexchange.com, janky-ass basestation.sqb's that have been - passed around for a decade. -- [ ] Find a source of aircraft images we can use. -- [ ] Use Wikipedia graph to rank landmarks? More inbound links to a - page means it has higher significance? -- [ ] Add support for Mastodon/ActivityPub. -- [ ] Generate our own maps instead of using adsbx's? The adsbx map - isn't always in sync with the data from the API. -- [ ] Run at global scale!