The original purpose of this script is described in my writeup for the Google Quickdraw visualizer that I created.
https://github.com/engelsjk/web-demo-quickdraw-visualizer
However, Google AI Experiments recently released another drawing tool, Autodraw. This new tool is a bit more functional than Quickdraw in that it tries to not only guess what you drew, but then offers to autocomplete your drawing by replacing it with a template of the same object. These templates or "stencils" are apparently curated by artists working with Google on these experiments.
I peaked behind the curtain of the new Autodraw tool and noticed some expected similarities to what I saw in Quickdraw.
First off, the Autodraw tool is firing off requests to the 'inputtools' API as you draw, similar to the Quickdraw tool.
https://inputtools.google.com/request?ime=handwriting&app=autodraw&dbg=1&cs=1&oe=UTF-8
The API base URL is the same as the Quickdraw tool and the query string parameters are similar with the exception of 'app=autodraw' instead of 'app=quickdraw'.
Additionally, the payload is similar in terms of structure, the primary difference being the "language":"autodraw" value. The "writing_guide" structure with the width/height of the drawing surface (or combined SVG elements?) and the "ink" array of x,y,t values is identical.
{"input_type":0,"requests":[{"language":"autodraw","writing_guide":{"width":389.99999999999994,"height":533},"ink":[[[x],[y],[t]]]]}]}
Out of curiousity, I ran the same ink array against the inputtools.google.com API, simply swapping out the app/language fields in each request. Somewhat surprisingly, the list of "guesses" and their associated scores are slightly different between apps. The list of scores below are the results from [Quickdraw, Autodraw], where a score of 0 indicates that the respective tool did not guess that object.
{u'belt': [8.4465789999999998, 5.1131152999999996],
u'ceiling fan': [9.6525490000000005, 0],
u'clarinet': [7.2248150000000004, 5.9660586999999996],
u'diving board': [9.4369300000000003, 7.1665105999999996],
u'dog': [9.8754220000000004, 6.2046099999999997],
u'drill': [9.0572569999999999, 0],
u'duck': [0, 7.3950110000000002],
u'eyeglasses': [0, 6.7490315000000001],
u'flamingo': [0, 7.5080523000000001],
u'frying pan': [0, 7.2749844000000001],
u'garden hose': [9.2187000000000001, 5.9557359999999999],
u'giraffe': [7.5582231999999996, 5.5352297000000004],
u'hockey stick': [9.5221999999999998, 0],
u'key': [9.8502120000000009, 7.0177546],
u'saxophone': [0.12961674000000001, 0.32083224999999999],
u'shovel': [7.1525097000000004, 5.5746016999999997],
u'snake': [9.4995639999999995, 7.6192570000000002],
u'snorkel': [2.9602336999999999, 3.4037480000000002],
u'sock': [7.8140916999999996, 3.8865327999999999],
u'spoon': [8.5633545000000009, 6.3431839999999999],
u'stethoscope': [0, 6.6366605999999999],
u'swan': [2.7948647000000002, 1.7449923000000001],
u'telephone': [8.0279875000000001, 0],
u'toilet': [8.9421189999999999, 0],
u'trombone': [5.4856075999999998, 4.6257023999999998]}
Additionally, I noticed that there's a list of 'stencils' that is loaded as a JSON file on load of the Autodraw tool.
https://autodraw.com/assets/stencils.json
This JSON file is a list of objects that appear to be the fixed set of guesses that can result from the machine-learning backend. For each object, there's a set of asset links to the SVG images that have been curated by various artists for this tool.
For example, one of the assets for the guess of 'lion' has a URL of https://storage.googleapis.com/artlab-public.appspot.com/stencils/selman/lion-01.svg
To date (Apr-14-2017), the stencils.json list only contains 371 objects, with between 1-19 assets for each object.