Skip to content

Elastic Search & Lucene

flbulgarelli edited this page Dec 5, 2014 · 2 revisions

To index or not to index

You index a field if you intend to search on it. You can only index text fields.

Options for indexing

  • Analyzed: breaks each field's value into a collection of tokens. This is what you need if you want the field to be full text searchable. If you also need it to be sortable, you'll have to create a not analyzed mirror of it.

  • Not analyzed: use this index option if you need exact match searching on the field. You can only sort by a text field if it's not analyzed.

  • No: you don't index, so you won't be able to search on the field.

Managing Elastic Search (ES) from Resource Map (RM)

We use ElasticSearch-Ruby as an interface from our Ruby codebase to our ES storage. To keep our ActiveRecord models tidy, we're using ActiveSupport concerns, which hold all methods and attributes relevant specifically to ES. We store Collections, Sites and Fields on ES. Each of these types has its own ES Concern: Colllection::ElasticsearchConcern, Site::ElasticsearchConcern and Field::ElasticsearchConcern.

Mappings

ES Mappings let you specify how each field in a document is stored and analyzed. If you want to implement a feature involving new search or sort capabilities, you'll probably need to modify some of the ES mappings used by RM.

RM's primary storage is MySQL. When there's a need to provide search features involving any type of entity (such as Collections, Sites and Fields) we add them to ES while still keeping them in MySQL. Thus, we need some sort of mechanism to keep ES up to date with what's in MySQL.

Collection::TireConcern defines after_create and after_destroy filters to achieve that. The method create_index, called after a collection is created, defines the mappings used to store collections in ES. Since collections contain sites, create_index makes use of Site::IndexUtils.site_mapping to generate its sites' mappings. A site's fields' mappings are defined at Field::TireConcern.index_mapping.

To sum up, if you need to modify mappings, this is where you should look at:

- Collection mappings are defined at Collection::TireConcern.create_index - Site mappings are defined at Site:IndexUtils.site_mapping - Field mappings are defined at Field::TireConcern.index_mapping

References

Lucene in Action