Adding additional optional args for decoding flags and AutoModel kwargs to support models like ReplitLM

[madhavatreplit]

Why

We require the ability to configure the tokenizer.decode call, as well as model args in the AutoModelForCausalLM.from_pretrained to support models like ReplitLM.

What changed

We add two input arguments with safe default behaviour to the main.py script:

1. clean_up_tokenization_spaces : bool

• this boolean flag is passed to tokenizer.decode to prevent tokenization spaces from being cleaned up. This flag affects spacing and therefore syntax in generated code with certain tokenizers such as the ReplitLM tokenizer.
• defaults to True, stores False

2. automodel_kwargs: json.loads, aka. a "stringified" JSON

• a "stringified" JSON that sets what default config values should be overriden in this harness to reproduce results.
• updates default init config key-values by being passed into the AutoModelForCausalLM.from_pretrained as kwargs. See the logic of why and how this works here in the transformers documentation.
• defaults to empty stringified JSON: "{}".

Rollout

[x] This is fully backward and forward compatible

[madhavatreplit]

This one should be checked for correctness by maintainers. Earlier was set to False, but with the flag being passed, the default behaviour is now conditioned on the flag.

More explicitly, if a user does not supply the arg --clean_up_tokenizations=False the expected default behaviour of this code path changes.

[loubnabnl]

Actually most models (including replit) and tasks decoding happen here with clean_up_tokenizations as False because eos_token is in stop_words whenever it exists. I suggest we change the argument definition so it stores False and becomes True when called by the user. For the else scenario, default behavior will change but I don't think that impacts performance.

[loubnabnl]

Thanks for the PR! clean_up_tokenization_spaces is already set to False during decoding for most tasks and models including replit model, so I suggested below that we keep it as False by default and True if a user specifies the argument.
The automodel_kwargs additions looks good.

[loubnabnl]

Actually most models (including replit) and tasks decoding happen here with clean_up_tokenizations as False because eos_token is in stop_words whenever it exists. I suggest we change the argument definition so it stores False and becomes True when called by the user. For the else scenario, default behavior will change but I don't think that impacts performance.

[loubnabnl]

Suggested change

	action="store_false",
	help="Set the clean_up_tokenization_spaces in tokenizer.decode() to False for specific models, defaults to True",
	action="store_true",
	help="Set the clean_up_tokenization_spaces in tokenizer.decode() to True for specific models, defaults to False",

see comment above