Merge branch 'develop' of github.com:meta-toolkit/meta into develop

CHANGELOG.md

@@ -1,3 +1,44 @@
+# [v2.4.0][2.4.0]
+## New features
+- Add a minimal perfect hashing implementation for `language_model`, and unify
+  the querying interface with the existing language model.
+- Add a CMake `install()` command to install MeTA as a library (issue #143). For
+  example, once the library is installed, users can do:
+
+    ```
+    find_package(MeTA 2.4 REQUIRED)
+
+    add_executable(my-program src/my_program.cpp)
+    target_link_libraries(my-program meta-index) # or whatever other libs you
+    need from MeTA
+    ```
+- Feature selection functionality added to `multiclass_dataset` and
+  `binary_dataset` and views (issues #111, #149 and PR #150 thanks to @siddshuk).
+
+  ```cpp
+    auto selector = features::make_selector(*config, training_vw);
+    uint64_t total_features_selected = 20;
+    selector->select(total_features_selected);
+    auto filtered_dset = features::filter_dataset(dset, *selector);
+  ```
+- Users can now, similar to `hash_append`, declare standalone functions in the
+  same scope as their type called `packed_read` and `packed_write` which will be
+  called by `io::packed::read` and `io::packed::write`, respectively, via
+  argument-dependent lookup.
+
+## Bug fixes
+- Fix edge-case bug in the succinct data structures
+- Fix off-by-one error in `lm::diff`
+
+## Enhancements
+- Added functionality to the `meta::hashing` library: `hash_append` overload for
+  `std::vector`, manually-seeded hash function
+- Further isolate ICU in MeTA to allow CMake to `install()`
+- Updates to EWS (UIUC) build guide
+- Add `std::vector` operations to `io::packed`
+- Consolidated all variants of chunk iterators into one template
+- Add MeTA's citation to the README!
+
 # [v2.3.0][2.3.0]
 ## New features
 - Forward and inverted indexes are now stored in one directory. **To make
@@ -435,7 +476,8 @@
 # [v1.0][1.0]
 - Initial release.
 
-[unreleased]: https://github.com/meta-toolkit/meta/compare/v2.3.0...develop
+[unreleased]: https://github.com/meta-toolkit/meta/compare/v2.4.0...develop
+[2.4.0]: https://github.com/meta-toolkit/meta/compare/v2.3.0...v2.4.0
 [2.3.0]: https://github.com/meta-toolkit/meta/compare/v2.2.0...v2.3.0
 [2.2.0]: https://github.com/meta-toolkit/meta/compare/v2.1.0...v2.2.0
 [2.1.0]: https://github.com/meta-toolkit/meta/compare/v2.0.1...v2.1.0

PACKAGING.md

@@ -0,0 +1,38 @@
+# Packaging or Installing MeTA
+
+Please follow these slightly modified build instructions if you are either
+a packager and you want to make a package of MeTA for your distribution, or
+if you simply want to install MeTA to e.g. `/usr/local`.
+
+You will need the following `cmake` flags:
+
+- `-DBUILD_SHARED_LIBS=On`: build `.so` libraries instead of `.a`.
+- `-DBUILD_STATIC_ICU=On`: force building a static ICU library as part of
+    `meta-utf.so` to ensure Unicode standard stability.
+
+If you are building for a Linux platform, you *must* use GCC. **Do not use
+Clang/libc++**, as the library throws exceptions and must be built in the
+same way all other C++ applications on the system are expected to be built.
+
+MeTA will require [cpptoml][cpptoml] to be installed. This is
+straightforward as cpptoml is header-only. Please follow the standard
+`cmake` build instructions for cpptoml and package using `make install` to
+install the CMake configuration files for cpptoml. This allows MeTA's CMake
+configuration files to find cpptoml as a dependency.
+
+MeTA can then be installed using `make install`.
+
+# Using an Installed MeTA
+
+You can consume MeTA most easily by using a CMake build system (though you
+do not have to). Your `CMakeLists.txt` might look something like the
+following:
+
+```cmake
+find_package(MeTA 2.4 REQUIRED)
+
+add_executable(my-program my_program.cpp)
+target_link_libraries(my-program meta-index) # or any other MeTA libraries
+```
+
+[cpptoml]: https://github.com/skystrife/cpptoml

README.md

@@ -609,6 +609,12 @@ and related libraries significantly easier than it would be otherwise, and
 it tends to have very up-to-date packages relative to other similar MinGW
 distributions.
 
+**Note:** If you find yourself confused or lost by the instructions below,
+please refer to our [visual setup guide for
+Windows](https://meta-toolkit.org/windows-setup-guide.html) which includes
+screenshots for every step, including updating MSYS2 and the MinGW-w64
+toolchain.
+
 To start, [download the installer][msys2] for MSYS2 from the linked
 website and follow the instructions on that page. Once you've got it
 installed, you should use the MinGW shell to start a new terminal, in which

RELEASING.md

@@ -0,0 +1,58 @@
+# Releasing MeTA Versions
+
+This document contains a checklist for releasing a version of MeTA so we
+follow a consistent releasing process.
+
+1. Pick a version number. MeTA releases (Major.Minor.Patch) ought to be API
+   compatible with other releases that share the same Major and Minor
+   version numbers but different Patch versions.
+
+   Major API changes (like new libraries or toolkit-wide backwards
+   incompatible API changes) increment the Major release number. Minor API
+   changes (like enhancements) increment the Minor release number. Patch
+   versions should be released only for bug fixes.
+
+2. Ensure `CHANGELOG.md` is up to date.
+
+   If there are *any* breaking changes, mention these explicitly. If there
+   are migration strategies to work around these breaking changes, provide
+   a brief explanation (or a link to explain them).
+
+3. If there are major *or* minor API changes, ensure that the documentation
+   on the website (meta-toolkit/meta-toolkit.org) is correct.
+
+   Update Doxygen as necessary.
+
+4. Ensure that the build is passing on both Travis (Linux + OS X) and
+   Appveyor (Windows/MinGW-w64).
+
+5. Merge branch `develop` into `master` with a commit message
+
+   > Merge branch 'develop' for MeTA vX.Y.Z
+
+   Use `git merge develop --no-ff` to create a merge commit.
+
+6. Tag the merge commit. The tag should be both annotated *and* signed:
+
+   ```
+   git tag -as vX.Y.Z
+   ```
+
+   The tag message should be the contents of the CHANGELOG for that
+   version. Remove unnecessary markdown syntax like header markers and code
+   blocks. Backticks can stay.
+
+7. Push the merge and the tags to GitHub:
+
+   ```
+   git push --follow-tags
+   ```
+
+8. Create a release on GitHub using the new tag. Its title should be "MeTA
+   vX.Y.Z".
+
+   The contents of the message should be exactly the same as the CHANGELOG
+   entry for that release.
+
+9. Upload the model files and include a section in the GitHub release notes
+   containing their sha256 sums.