Ariel - A Ruby Information Extraction Library

Ariel intends to assist in extracting information from semi-structured documents including (but not in any way limited to) web pages. Although you may use libraries such as Hpricot or Rubyful Soup, or even plain Regular Expressions to achieve the same goal, Ariel approaches the problem very differently. Ariel relies on the user labeling examples of the data they want to extract, and then finds patterns across several such labeled examples in order to produce a set of general rules for extracting this information from any similar document.

When working with Ariel, your workflow might look something like this:

1. Define a structure for the data you wish to extract. For example:

    @structure = Ariel::StructureNode.new do |r|
      r.item :article do |a|
        a.item :title
        a.item :author
        a.item :date
        a.item :body
      end
      r.list :comments do |c|
        c.list_item :comment do |c|
          c.item :author
          c.item :date
          c.item :body
        end
      end
    end
   

2. Label these fields in a few example documents (normally at least 3). Labels are in the form of <l:label_name>…</l:label_name>
3. Ariel will read these examples, and try to generate suitable rules that can be used to extract this data from other similarly structured documents. Use Ariel#learn to initiate learn ruling.
4. A wrapper has been generated - we can now happily load documents with the same structure (normally documents generated by the same rules, so different pages from a single site perhaps) and query the extracted data. See Ariel#extract.