Eviction doesn't happen at all if recent_budget=0

[foreverpiano]

From

H2O/h2o_hf/utils_hh/modify_llama.py

Lines 140 to 156 in 281ffef

	attn_mask = torch.ones(current_scores_sum.shape[0], current_scores_sum.shape[1]+1).to(dtype_attn_weights).to(attn_weights_devices)
	attn_tokens_all = self.previous_scores.shape[-1]
	if attn_tokens_all > self.cache_budget:
	# activate most recent k-cache
	if not self.recent_budget == 0:
	attn_mask[:, :-self.recent_budget] = 0
	selected_set = self.previous_scores[:, :-self.recent_budget]
	else:
	# activate historical best self.cache_budget - self.recent_budget tokens.
	# self.previous_scores # (k-Cache - 1)
	selected_set = self.previous_scores
	if not self.heavy_budget == 0:
	_, keep_topk = selected_set.topk(k=self.heavy_budget, dim=-1, largest=True)
	attn_mask = attn_mask.scatter(-1, keep_topk, 1)

If recent_budget=0:
the mask set to one at first, attn_mask = torch.ones;
but scatter to one attn_mask = attn_mask.scatter(-1, keep_topk, 1);

attn_mask[:, :-self.recent_budget] = 0 only works for recent_budget !=0