utf-8 unicode

[haditabealhojeh]

Hi everybody. I am new to Norconex. I am trying to write a crawler using this config. but i got unicode characters instead of utf-8 (Persian) in the results.

[ohtwadi]

Please try with pretty=true when querying ES - http://localhost:9200/dental2-index/_search?pretty=true

The JSONFileCommitter clearly shows the characters as expected.

[{"upsert":{"reference":"https://www.paziresh24.com/app","metadata":{"title":["دانلود اپلیکیشن نوبت دهی پزشکان پذیرش24"]},"content":"دریافت نوبت\n\n         اولین نوبت خالی برای شماست! مهم نیست چه زمانی و کجا نیازی به پزشک داشته ... }}]

You may also want to read up on setting up your ES for the Persian language (see here).