-
Notifications
You must be signed in to change notification settings - Fork 10
Lukas
Lukas Schmelzeisen edited this page Aug 27, 2013
·
4 revisions
- Finding and documenting requirements for componentSearch
- Designing REST-API for search requirements.
- Designing Solr
schema.xmlfor metalcon intern data. - Learning how to create additional (Solr/Lucene) search indices on the fly, testing how performant this is.
- Finding a datastore and learning how to access it for metalcon intern data.
- Implementing API as a Java-Servlet.
- Implementing query Spellchecking / Suggesting.
- Implementing faceting.
- Developing a ranking algorithm for sorting search results.
- Installing technologyZooKeeper + technologyHadoop + technologyHBase.
- Installing, configuring and using technologySolr.
- Installing, configuring and using technologyNutch.
I'm maintaining this list for myself to get an overview over the work completed. It might become more/less detailed over time.
| Day | Time | Description |
|---|---|---|
| 2013-07-16 | 17:00-20:00 | Installing SSH, learning Git(hub) |
| 2013-07-21 | 21:00-00:00 | Learning GIT (game branching game) |
| 2013-08-22 | 10:00-18:00 | Wiki: clearing up requirements, adding components |
| 2013-07-30 | 21:00-22:00 | Editing Wiki, Setting up XMPP/Mumble |
| 2013-08-01 | 11:00-18:30 | Learning Nutch, Solr |
| 21:00-00:00 | Started creating a simple web crawler / full text seacher. | |
| 2013-08-02 | 11:00-19:00 | Doing a simple Nutch + Solr setup. |
| 2013-08-03 | 18:00-18:45 | Understanding Nutch configuration. |
| 2013-08-05 | 11:00-18:00 | Expanding on Nutch knowledge. |
| 2013-08-06 | 11:00-19:30 | Understanding HBase. |
| 2013-08-07 | 11:00-17:30 | Working on a Zookeeper+Hadoop+HBase+Tomcat+Solr+Nutch setup. |
| 2013-08-08 | 11:00-21:00 | Finishing techStack setup. |
| 2013-08-09 | 11:00-17:30 | Doing Nutch configuration. |
| 2013-08-12 | 11:00-17:00 | Reading Solr Reference Guide (mainly understanding Solr architecture + Schema.xml). |
| 2013-08-13 | 11:00-18:30 | Reading Solr Reference Guide (Query Syntax). |
| Creating basic componentSearch requirements. | ||
| Creating a basic overview of Solr in technologySolr. | ||
| 2013-08-19 | 15:00-18:15 | Getting up to date with components. Creating a search servlet frame. |
| 2013-08-20 | 13:00-17:40 | Fixing Patrick's HBase setup. |
| Importing Rene's crawl data of the metal web. | ||
| Learning JSON-simple Java library. | ||
| 21:20-23:20 | Some basic code refactoring for the searchServer. | |
| Implementing JsonPrettyPrinter. | ||
| 2013-08-21 | 10:40-12:00 | Refining JsonPrettyPrinter. |
| Adding Test Cases. | ||
| 15:40-17:15 | Implementing Highlighting. | |
| 2013-08-22 | 11:00-17:15 | Upgrading to Solr 4.4.0 (from Solr 4.3.1). |
| Getting an overview of Solr configuration. | ||
| Configurating our solrconfig.xml accordingly. | ||
| Switching solr.xml configuration for legacy mode to core discovery. | ||
| Documenting what I know about Solr. | ||
| 2013-08-23 | 13:15-XX:XX | Reworked requirments of search component. |
| Documented what I currently know about Solr in technoloySolr. |