Getting appointments using bash

Retrieving GNIB appointments using a bash scripr
published: (updated: )
by Harshvardhan J. Pandit
is part of: GNIB Appointments
bash GNIB python script

recap: In the previous post, I managed to get the available appointments by pasting a bit of javascript that used GET to retrieve the available appointments through the browser console.

source: hosted on Github here

webapp: hosted on Heroku here shows available timings for GNIB and Visa appointments

In this post, I describe further efforts that led to a bash / python script that pulled the appointments. The advantages of a script is that it can be automated and run anywhere. So there are possibilities such as running the script on a server every 30 mins and then sending an email or some form of notification when a new appointment becomes available.

The script should not be too complicated and must be as simple as possible to run. Ideally something like this (actual output):

$harsh@XNMPRO:scripts >./query_gnib_appointments.py
4 July 2017 - 10:00
21 July 2017 - 08:00

script

Bash is the defacto language scripts are written in (usually) and these days so is python. I also happen to have taken a liking to python, which is why I tend to use it for pretty much everything.

making the request

That being said, bash uses curl which is an utility to make web requests. The URL from the previous post was https://burghquayregistrationoffice.inis.gov.ie/Website/AMSREG/AMSRegWeb.nsf/(getAppsNear). Suffixing it with the required GET parameters, we get http://burghquayregistrationoffice.inis.gov.ie/Website/AMSREG/AMSRegWeb.nsf/(getAppsNear)?openpage=&dt=&cat=Study&sbcat=All&typ=Renewal. For brevity, I'm going to use just URL instead of the large wad of characters.

SSL verification

Using the redirect/location flag -L, the output produced was:

$harsh@XNMPRO:scripts >curl -L "URL"
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

I'm not much knowledgeable about how SSL works at this level, so I just digged through the manpage to find the -k flag which disables SSL protection. Additionally, I used the -s flag to suppress progress bars from polluting the output. Using it yielded the response required:

$harsh@XNMPRO:scripts >curl -L -k "URL"
{"slots":[{"time":"21 July 2017 - 08:00", "id":"5D2DE14F0B06EFF8802581250031862F"}]}

Parsing JSON

If you can get away with manually eyeballing the response to get available appointment times, great. But I need precisely just the appointment timings. Which meant that I needed some way to parse that JSON response. So many tools, and all of the pre-installed languages, all have JSON handling libraries. Take your pick. I found a good variety on stackoverflow.

jq

Working in shell scripts, jq is a great tool for working with JSON. It can be used as simply as:

$harsh@XNMPRO:scripts >curl -L -k "URL" | jq ".slots | .[] | .time"
"21 July 2017 - 08:00"

jq uses syntax for selecting elements from JSON. The .slots instructs it to extract the value at that particular key. | (pipe) works as a filter and passes the data extracted from one part onto the next. .[] is an array operator, it acts as a for ... each on the array elements. .time then extracts the value for the appointment and prints it out.

grep

Using pattern matching, it is possible to only print out those lines that contain the word "time".

$harsh@XNMPRO:scripts >curl -L -k "URL" | grep -Po '"time":.*?[^\\]",'
"time":"21 July 2017 - 08:00",

python

After those two, it seems a bit silly to use python, but that's what I did anyway. In python, it is possible to execute a script inline using the -c command. The full script to parse out this particular JSON is:

import sys
import json
data = json.load(sys.stdin)
if data.get("error", None) is not None:
    raise Exception("ERROR: %s" % data["error"])
# If there are no appointments, then the empty key is set
if data.get("empty", None) is not None:
    print("No appointments available")
    sys.exit(0)
# There are appointments, and are in the key "slots"
data = data.get("slots", None)
if data is None:
    raise Exception("Data is NULL")
# This should not happen, but a good idea to check it anyway
if len(data) == 0:
    print("No appointments available")
    sys.exit(0)
# print appointments
# Format is:
# {
#   "id": "str",
#   "time": "str"
# }
for appointment in data:
    print(appointment["time"])

To use this, just put everything between triple-quotes ''' as this constitutes a multiline comment/statement in python.

$harsh@XNMPRO:scripts >curl -L -k "URL" | python '''
# program goes here
'''

Future work

  • Python script to do everything from request to parsing
  • Chrome extension of some sorts
  • Put the script up on a website for easier access to appointments