documentation

doc/cbird-devel.md

@@ -5,7 +5,7 @@ Release Blockers (v0.7)
 Wish List
 =========================
 
-### Command-Line
+### Command Line
 - "-use" special notation to find root index in the parent (tree)
 - symbolic names for all enumerations (e.g. "v" == video)
 - disable/reduce logging (-verbose, -quiet etc)
@@ -20,27 +20,27 @@ Wish List
 - -with type <x> , recognize enumerations as in params
 
 ### Indexing
+- file/directory name filters for inclusion/exclusion
 - store date-modified,size for better updating
 - also store container md5 (.zip) for faster updating/verifying
-- >64k frames per video
+- \>64k frames per video
 - capture more common errors in indexer
   - unsupported ICC profile
   - colordescriptor on grayscale image
 - console progress bar
 - error-log to file
 - store symlinks to prevent broken links later on
-- use idct scaling to speed up jpeg decompress (wip)
-- file/directory name filters for inclusion/exclusion
+- ~~use idct scaling to speed up jpeg decompress (wip)~~
 - ~~hard links handling~~ added in v0.6
   - ~~isJunction() exclusion from index~~
   - ~~exclusion from index (map inodes during scan)~~
 - ~~see if `skip_loop_filter` for h264 decoding is a good idea: about 20% faster decoding, unknown affect on hash quality~~ enabled in v0.6
 - ~~same with `SWS_AREA` rescaler instead of `SWS_BICUBIC`~~ enabled in v0.6
-- index videos with partition and merge approach to overcome ffmpeg limitations
+- index videos with partition and merge approach to overcome ffmpeg thread limitations
 
 ### Search
 - search tree for histograms
-- csv output for other tools, gui wrappers
+- csv/json output for other tools, gui wrappers
 - sort groups with closest matches first
 - fast block-averaging template match with threshold
 - ~~filter for hard/soft links~~ added in v0.6
@@ -59,10 +59,10 @@ Wish List
 - ~~remember past deletions and optionally replay them in the future should they reappear (via traal)~~ added "weeds" feature v0.6
 - detect breaking of symlinks on delete/rename
 - visual indicator of the needle in group view, gets lost when rotating
-- side-by-side playback: fix narrow videos
+- ~~side-by-side playback: fix narrow videos~~
 - video compare: use the same zoom/pan controls as image view
 - select-all, clear-selection
-- action groups to compact the context menu
+- ~~action groups to compact the context menu~~
 - option to force layout to use one row/column
 
 ### Misc
@@ -71,13 +71,15 @@ Wish List
 - unecessary "virtual"
 - "override"
 - "const"
+- replace getenv() calls with qt version
 
 Major Bugs
 ==========================
 - might be possible for -remove to corrupt database
   - e.g. -select-type 1 -remove -update
 - sws_scale buffer overflow (264x480 yuv420p)
 - ~~MBW move folder option broken~~ removed in v0.5.1
+- control-c during database write will usually corrupt sqlite database, most likely when the filesystem doesn't support locking
 
 Minor Bugs
 =========================
@@ -99,23 +101,22 @@ Minor Bugs
 - MGLW: rename folder doesn't update all affected viewer paths
 - MGLW: difference image clips white/light shades of grayscale images
 - MGLW: mouse motion eats the next scroll wheel event  (gnome3+xcb)
-- replace getenv() calls with qt version
 - weeds: when deleting a file, do something about broken weeds condition
 - weeds: add something to report and fix broken weed records, maybe part of -update
-- Theme: background shade stacks up with context menu
+- ~~Theme: background shade stacks up with context menu~~
 - CropWidget: redraw ghosting selection rect, also x,y offset problems
-- Mac: native yes/no dialog has no shortcut keys and weird icon
+- Mac: native yes/no dialog has no shortcut keys ~~and weird icon~~
 - AppImage: Fedora: "EGL Not Available" and no window titlebar
   - fix: change qt platform `-platform wayland-egl` or xcb
 
 Theme
 =========================
 
-## qdarkstyle/*/ (QSS widget stylesheet)
+#### qdarkstyle/*/ (QSS widget stylesheet)
 - QDarkStyle has been modified to make it grayscale and fix a few issues, todo: fork on github and put the link here
 - tools/copystylesheet.sh copies Dark and Light style from QDarkStyle project
 
-## res/cbird.qss
+#### res/cbird.qss
 - contains colors for stuff
 
 #### res/cbird-richtext.css
@@ -148,7 +149,7 @@ Unit tests
 
 Cbird uses the QTest unit test framework. The tests require a compatible test data set, see release page on github.
 
-```
+```shell
 export TEST_DATA_DIR=</path/to/cbird-testdata>
 cd unit/
 

readme.md

@@ -2,39 +2,51 @@
 
 About cbird
 =========================
-cbird is a command-line program for finding duplicate images and videos that cannot be found by general methods such as file hashing. Rather, cbird employs Content-Based Image Recognition (CBIR) which examines the pixels of files to get comparable features and "perceptual" hash codes which can match other files with similar pixels.
+cbird is a command-line program for finding duplicate images and videos that cannot be found by general methods such as file hashing. Content-Based Image Recognition (CBIR) is used, which examines the pixels of files to get comparable features and "perceptual" hash codes.
 
 The main features are:
 
 - CBIR algorithms for image and video search
 - Cached index in the top directory, prevents needless re-indexing
 - GUI for evaluating duplicates
-- Huge index/database is possible without major slowdowns
+- Huge index/database is possible
+- Comprehensive file format support
 - Search inside zip files
 
 Installing
 =========================
 
+### Compile
+
+Compile it yourself using [my detailed notes](doc/cbird-compile.md).
+
 ### Download
 
 [Download binaries on Github](https://github.com/scrubbbbs/cbird/releases)
 
 #### Linux AppImage
 
+Add execute permission and run
+```shell
+chmod +x cbird-0.7.0-x86_64.AppImage
+./cbird-0.7.0-x86_64.AppImage -install # optional install helper
+cbird [...]
 ```
-chmod +x cbird-linux-0.5.0-x86_64.AppImage
-./cbird-linux-0.5.0-AppImage -install # optional install to /usr/local
-```
-
-- If missing libOpenGL.so.0:
-  + debian: apt install libopengl0
-  + redhat: yum install libglvnd-opengl
 - Required packages: trash-cli
 - Optional packages: ocenaudio, kdenlive
 
-#### Mac OS X 11.0+
+##### AppImage Issues
+- AppImage won't run (debian, Ubuntu 23.04+)
+  + `apt install libfuse2`
+- Missing libOpenGL.so.0
+  + debian: `apt install libopengl0`
+  + redhat: `yum install libglvnd-opengl`
+- Missing window titlebar (Fedora)
+  + `cbird -platform wayland-egl [...]` 
+
+#### Mac OS X 11.0+ x86
 
-- Intel CPU: Unzip the distribution file and run:
+- Unzip the distribution file and run
 
 `cbird/cbird-mac [...]`
 
@@ -44,8 +56,7 @@ chmod +x cbird-linux-0.5.0-x86_64.AppImage
 - Install helpers (optional): vlc, kdenlive
 
 ##### Windows PowerShell
-
-Optional: for setting up shortcuts to cbird commands
+Optional: create shortcuts for cbird
 
 - Unzip into C:\ so you have C:\cbird\cbird.exe
 - Enable script execution
@@ -65,9 +76,9 @@ Getting Started
 ========================
 
 #### Get Help
-`cbird -help` [is very detailed](https://gist.github.com/scrubbbbs/4c65058c054822d4f279c00f53ddd3aa)
+`cbird -help` [is very detailed](https://gist.github.com/scrubbbbs/c7177a0a97e098c6bb10ae4afe8f6c35)
 
-#### Index the files in `<path>`, caching into <path>/_index
+#### Index the files in `<path>`, caching into `<path>/_index`
 `cbird -use <path> -update`
 
 #### Index files in cwd
@@ -162,12 +173,19 @@ There are a few for power users.
 - `CBIRD_CONSOLE_WIDTH` set character width of terminal console (default auto-detect)
 - `CBIRD_COLOR_CONSOLE` use colored output even if console says no (default auto-detect)
 - `CBIRD_FORCE_COLORS` use colored output even if console is not detected
-- `CBIRD_LOG_TIMESTAMP` add time delta to log mes	sages
-- `CBIRD_NO_BUNDLED_PROGS` do not use bundled programs like ffmpeg in the appimage****
+- `CBIRD_LOG_TIMESTAMP` add time delta to log messages
+- `CBIRD_NO_BUNDLED_PROGS` do not use bundled programs like ffmpeg in the appimage/binary distribution
 - `QT_IMAGE_ALLOC_LIMIT_MB` maximum memory allocation for image files (default 256)
 - `QT_SCALE_FACTOR` global scale factor for UI
 - `TMPDIR` override default directory for temporary files; used for opening zip file contents
 
+Wish List, Bugs, Etc
+====================
+
+Check the [development notes](doc/cbird-devel.md) for known bugs and feature ideas
+
+Report bugs or request features [on github](https://github.com/scrubbbbs/cbird/issues)
+
 Search Algorithms
 ====================
 There are several algorithms, some are better than others depending on the situation.
@@ -259,36 +277,59 @@ Arguments | Note | Time (s)
 -----|------|------ 
 -p.alg dct -p.dht 2  | dct, threshold 2           | 1.3
 -p.alg fdct -p.dht 7 | dct-features, threshold 7  | 1.5
--p.alg orb           | orb-features               | 84.4**
--p.alg color         | colors                     | dnf***
-
-**OpenCV search tree only partially cached on disk, slow to start
-***Color search lacks a search tree, not suitable for large sets
+-p.alg orb           | orb-features               | 84.4[^1]
+-p.alg color         | colors                     | dnf[^2]
 
 Release Notes
 =============
 
-#### v0.6.3
-
-- Image views are display-dpi aware (100%==actual pixels)
-- Use idle process priority when indexing
-- Update for current FFmpeg (60)
-- Much faster jpeg indexing using idct scaling
-- Sort filenames correctly when containing numbers
-- Functions: make chainable and consistent in all contexts
-- Dark/Light theme included
-- Mac OS X support
-
-#### v0.6.2
-
-- Qt 6 port
-- Thumbnail cropping tools
-- Search: Query external hash code
-- Results view: Resize page option
-- Folder view: Fix video thumbnail size
-
+#### v0.7
+
+- Mac OS X port and x86 binary package
+- Use current libraries (FFmpeg, Qt6, Quazip)
+- Command Line
+  - describe what -filter is doing
+  - cleanup some logging
+- Indexing
+  - Use "nice"/low priority when scanning/indexing
+  - 3-4x faster jpeg indexing using idct scaling
+  - Recognize more video extensions
+  - Remove colon (":") workaround for powershell
+  - Failsafe cache updates using write-temporary-move method
+  - Fix slow cache update on network volumes
+  - Fix -i.ignored to actually track all ignored stuff
+- Searching
+  - Add % progress for sql queries, cleanup logging
+  - Search using a given hash code
+  - Use numeric sort for filenames (e.g. 1.jpg sorts before 10.jpg)
+  - Functions are always chainable and consistent between commands
+  - Properties: add aspectRatio
+  - Disable inode check on win32, too slow on network volumes
+  - Fix attempt to template match video result
+- Viewing
+  - Add theme system, remove hardcoded theme/style
+	- auto-detect dark/light theme on mac/windows
+	- shade background windows when modal window is displayed
+	- "-theme" to change the theme
+  - Streamline context menu
+  - ffplay-sbs: tons of improvements
+  - Do not require index for -select-files
+  - DPI-aware image scaling
+  - Show creation time metadata for videos too
+  - Browse the selected item in its folder
+  - Use grid for folder view and fixed thumbnail aspect ratio
+  - Cycle contents thumbnails when hovering in folder view
+  - Add index/folder thumbnail tool
+  - Fix some performance issues
+  - Fix keyboard navigation in folder view
+  - Fix missing match range
+  - Prevent crash/hang on some invalid inputs (corrupt videos etc)
+  - Fix slow qualityscore on network volumes
+  - Fix potential crash on two-tone images
+  - Fix potential crash/hang with corrupt video files
+  - Fix elided strings display
+
 #### v0.6.0
-
 - Add weeds feature (recommended by r/user/traal)
 - Search: Add max-threshold `-p.mt` which produces a result for every needle until threshold is exceeded
 - Indexer: Use skip-loop-filter to speedup video indexing ~20%
@@ -299,3 +340,6 @@ Release Notes
 - Video view: fix issues with unequal frame rates
 - Video view: export the aligned pair to Kdenlive
 - Speedups: video compare view, differences image, `-group-by`
+
+[^1]: OpenCV search tree only partially cached on disk, slow to start
+[^2]: Color search lacks a search tree, not suitable for large sets

res/usage.txt

@@ -46,7 +46,7 @@ Selections
   -select-result                   to chain queries: convert the last query result into a selection, and clear the result
   -select-sql <sql>                items with sql statement [select * from media where ...]
   -select-files <file> [<file>]... ignore index, existing files of supported file types
-  -select-grid <file>              ignore index, detect a grid of thumbnails,  eak up into separate images
+  -select-grid <file>              ignore index, detect a grid of thumbnails, break up into separate images
 
 Batch Deletion
 ==============================================================================
@@ -88,8 +88,8 @@ Viewing
 ==============================================================================
   -folders                         enable group view, group results from the same parent directory
   -sets                            enable group view, group results with the same pair of directories
-  -exit-on-select                  \"select\" action exits with selected index as exit code, < 0 if canceled
-  -max-per-page <int>              maximum items on one page [10]
+  -exit-on-select                  "select" action exits with selected index as exit code, < 0 if canceled
+  -max-per-page <int>              maximum items on one page [12]
   -show                            show results browser for the current selection/results
   -theme <string>                  select color theme, must appear before -show
      Auto                          * detect Dark/Light from system theme if possible (Mac/Windows default)