Collecta XMPP API Ruby Library

Ruby library for easy working with the Collecta XMPP API.

Required gems and libraries

• xmpp4r - low level XMPP manipulations
• crack - XML parsing
• switchboard - only for collecta_jack.rb
• xmpp4-simple - only for collecta_bot.rb
• eventmachine - only for collecta_bot.rb
• sinatra - only for collecta_web.rb
• json - only for collecta_web.rb

Implemented features

collecta.rb

• Jabber::PubSub::ServiceHelper::subscribe_to_with_options() function for passing Data Forms option on node subscription
• Collecta::Client (inheriting Jabber::Client) for SASL ANONYMOUS connection, PubSub (un)subscription etc.
• Collecta::Payload class, encapsulating Collecta Messages (XML to Hash conversion is done via Crack

collecta_jack.rb

• Switchboard::AnonymousClient (inheriting Switchboard::Client) for SASL ANONYMOUS connection
• Switchboard::CollectaClient (inheriting Switchboard::AnonymousClient) and adding Collecta default settings (service URL, node etc.)
• CollectaJack - Switchboard Jack for PubSub subscription etc.

test_search.rb and test_jack.rb for library testing and demonstration.

collecta_bot.rb

• Collecta::Task - EventMachine::Deferrable - based - background processing for long-running tasks
• Collecta::Bot - EventMachine -based XMPP bot, sending results from the Collecta searches to some JID

collecta_web.rb

• Collecta::App - Sinatra-based simple web API (inheriting Sinatra::Default)

Usage

Download the library from the GitHub repository and copy all files inside your project's sources directory. Be sure to pass your Collecta API key on the Collecta::Client.connect() function invoking.

Working directly with xmpp4r

require 'collecta'

apikey = "..."
query, notify = "iphone", "apple, mac"

@client = Collecta::Client.new(apikey)
@client.anonymous_connect
# search for 'iphone' and notifications for 'apple' and 'mac'
@client.subscribe(query, notify)
@client.add_message_callback do |msg|
    next unless payload = Collecta::Payload.new(msg)
    # do something with the messages
end
...
at_exit do
    @client.unsubscribe
    @client.close
end

Working with switchboard

require 'collecta_jack'

settings = YAML.load(File.read("config.yml"))
# query from the command line, no notifications, debug enabled
switchboard = Switchboard::CollectaClient.new(settings["collecta.apikey"], ARGV.pop, nil, true)

switchboard.on_collecta_message do |msg|
    payload = Collecta::Payload.new(msg)
    # do something with the messages
end
 
switchboard.run!

XMPP Bot

Available Bot Commands

• HELP - just a commands list. TODO: better help message with commands description
• PING - check the connection. Will ask also for authentication
• S, s - subscribe to some query. Will start sending the results to the console JID ("bot.console")
• N, n - subscribe to notify. TODO: announce the notify results only when ask for them
• UN, un, U, u - unsubscribe from ALL subscriptions

Make sure you have the correct settings for your bot JID and password ("bot.jid" and "bot.password") and for the account, that will receive all messages ("bot.console")

require 'collecta_bot'

class SearchBot < Collecta::Bot
    def self.format_result(msg, debug = true)
        # some fancy message formatting
        ...
    end
end

SearchBot.run("config.yml")

Desktop notifications (Growl, libnotify)

collecta_notifyio.rb

Desktop notifications via http://notify.io/ service are also available. You can use the original Growl client for Mac or my python-notify based client for Linux.

Be sure to adjust notifyio.userhash and notifyio.apikey in the config file. You can get their values from the notify.io settings page.

You need to supply the search query via the command line parameter:

$ ruby ./collecta_notifyio.rb "iphone category:story"

Simple web API (Sinatra application)

collecta_web.rb

The API is not designed to be full featured web service. His primary goal is to be a backend for the main web service, deployed on Google AppEngine or Heroku. The main service need to take care for the users authentication and management, saving queries per user etc. But because most of the deployment environments (GAE, Heroku etc.) cannot work with XMPP PubSub, the current simple web API will be used for the real requests to the Collecta XMPP API. The only protection in the moment is the sig parameter (signature), send with each request. The signature is generated as follows:

sig = Digest::SHA1.hexdigest("--#{CFG['web.apikey']}--#{jid}") 
# example
web.apikey = "SomeSecret"
jid = "[email protected]"
sig = "3dbc8601b2fd476b3eee59814d1a6ba8328e5492"

On the server side web.apikey must be set inside the config.yml . For example:

# --- config.yml
....
web.apikey: SomeSecret
bot.jid: [email protected]
....

The API contains just two POST requests methods ( /1/sub/ and /1/unsub/ for subscription to some query and unsubscription from all queries) and one GET request method ( /1/list ) for some JID subscriptions listing.

Required parameters:

• jid - JID on the account, that will recieve the results from the search
• sig - signature - created as Digest::SHA1.hexdigest() of web.apikey from the config.yml configuration file and jid
• q - Collecta query string (only for subscriptions - /1/sub/ calls)

Optional parameters:

• callback - only for GET listing requests ( /1/list ). JSONP-ready

Starting the web API (it's Sinatra (means Rack) application) on port 8080:

$ cd collecta-xmpp/
$ rackup -p 8080

Example command-line usage:

// [email protected]  will recieve results from the 'iphone'  and 'mac category:story' searches
$ curl -X POST -d'sig=97...' -d'[email protected]' -d'q=iphone' http://example.com/1/sub/
$ curl -X POST -d'sig=97...' -d'[email protected]' -d'q="mac category:story"' http://example.com/1/sub/
// to which quieries is subscribed [email protected]
$ curl -v "http://example.com/1/[email protected]&sig=97..."
...
< Content-Type: application/json; charset=utf-8
...
["iphone","\"mac category:story\""]
// unsubscribe from all queries
$  curl -X POST -d'sig=97...' -d'[email protected]' http://example.com/1/unsub/

The subscribed JID ([email protected] in the example above) need first to include the "bot.jid" JID from the config.yml file ([email protected] in config.yml.dist) in his roster and authorize it for messages exchange.

collecta_shell.rb

Subscribing JIDs to some queries and unsubscription can be done also via a simple command-line shell. This application is independent from the web API. They does not share their users subscriptions. In both application that relations are kept only in the memory. The main web service need to take care for the saving of that relations.

Starting the shell:

$ ruby ./collecta_shell.rb 
> help
"sub JID QUERY - subscribe JID to QUERY"
"unsub JID     - unsubscribe JID"
"exit          - exits the app"
"help          - this help"
>

Subscribe/unsubscribe some JID to/from a query:

> sub [email protected] "iphone category:story"
> unsub [email protected]
> exit
Finished
$

ToDo

• ruby gem
• patched switchboard for command line parameters usage ( --query iphone --notify "apple,mac" )