Auto code completion rarely happens

[rmtew]

Auto completion rarely happens. Maybe I have seen it once unexpectedly. I have to Control+L to get the suggestions.

A contrived example for the sake of a small file I would hope to have it work within. I have a file y.c with the following contents:

#include <stdio.h>

void main() {
    FILE *f = fopen("file", "rb");
    
    fclose(f);
}

I close vscode. I reopen it. I move the cursor to the line after fopen then I wait. I see nothing. I can try and define new structs. I can try anything. The only way I get a response is Control+L for example showing it is working:

Windows 11 x64
VS Code 1.96.4 (just updated)
llama.vscode 0.0.4 (auto updated)
The setting for auto-completion is still the default and is enabled.

Thanks for the extension it has been nice to play with.

[hkr04]

Did you check the log of llama-server? If code completion is triggered, you should get the info like this:

slot launch_slot_: id  0 | task 431 | processing task
slot update_slots: id  0 | task 431 | new prompt, n_ctx_slot = 32768, n_keep = 0, n_prompt_tokens = 168
slot update_slots: id  0 | task 431 | kv cache rm [107, end)
slot update_slots: id  0 | task 431 | prompt processing progress, n_past = 168, n_tokens = 61, progress = 0.363095
slot update_slots: id  0 | task 431 | prompt done, n_past = 168, n_tokens = 61
slot      release: id  0 | task 431 | stop processing: n_past = 174, truncated = 0
slot print_timing: id  0 | task 431 | 
prompt eval time =     617.21 ms /    61 tokens (   10.12 ms per token,    98.83 tokens per second)
       eval time =     571.73 ms /     7 tokens (   81.68 ms per token,    12.24 tokens per second)
      total time =    1188.95 ms /    68 tokens
srv  update_slots: all slots are idle
request: POST /infill 127.0.0.1 200

[igardev]

Thanks for the observation and for the remark.
The autocomplete is triggered automatically if you type, delete, press enter to to insert a new line, etc. (i.e. change something). If you just move the cursor the extension will not try to make a suggestion. Ctrl+L (or Ctrl+Shift+L) triggers it manually. I think this behavior makes sense (Copilot follows the same logic as far as I know).

[ggerganov]

Thanks for the clarification. I updated the readme to say that the auto-completion is triggered on changes, not on movement: fa1d157

[rmtew]

The log is static for typing-based auto-complete. If it happens at all I am not sure. The only time I can reliably reproduce auto-complete is if it follows a manual invocation with Control-L and I type into it and modify it as I type. If I dismiss the auto-completion then it does not appear to restart until I do a manual invocation on an partially typed line where it can provide a valid completion to edit.

Here's my reproduction case:

1. Open the file as in the original report.
2. Put the cursor above main.
3. Move it to the line under FILE and type fseek. Nothing happens.
4. Press Control+L and get suggested fseek(f, 0, SEEK_END);. Take the suggestion. No change in behaviour on the next line.
5. Delete the line. Hit enter on the FILE line and then tab to get under FILE again.
6. Type fseek. Nothing happens.
7. Press Control+L and get suggested fseek(f, 0, SEEK_END);. Do not take it, but type into it seeing the auto-completion remaining. At SEEK_ type S and get offered SEEK_SET and TAB to get fseek(f, 0, SEEK_SET);

The plugin updated to 0.0.5 and I restarted extensions as prompted before this.

[igardev]

Thanks for the detailed explanations. Please, check the extension setting "Llama-vscode: Auto". If it is unchecked, you will never get automatic completions, only manual. If it is checked, but you still don't get completions on typing, there is a bug. I didn't manage to reproduce it, but will try again. By the way, in the status bar, near the text "llama-vscode", there is an indication what is happening. On starting a query to the llama.cpp server it says "thinking...", on getting a result it shows statistics (i.e. time for the request) and "no suggestion" if the model has no suggestion in the current context.

[rmtew]

With regard to the setting. From my original report "The setting for auto-completion is still the default and is enabled."

With regard to the status bar. Yes, it appears to reflect what I see happening IIRC. Manual prompting produces a result and changes in the bar text and when auto-completion is not happening but should it reflects inactivity.

[igardev]

Thanks for the patience. Very interesting case. I can't reproduce it for now, but will continue trying.

In the latest version there are 2 additional settings, which also affect completion: llama-vscode: Enabled (should be checked) and llama-viscode: Language Settings (should be * true). Probably they are not related to your problem, but you could check them.

You could try to uninstall the extension, restart VS Code and install the extension again. Not sure if this will help, but will not hurt.

I am afraid I don't have a good solution for your problem for now. If I find something will write it here.

[rmtew]

The two settings were set. Uninstalled, restarted and reinstalled did not make a difference.

If anyone else gets this until the reason is discovered just press Control+L on each line to get it working and it seems to stay working for just that line.

[ggerganov]

I wonder if this behavior could be caused due to a conflict with another extension somehow. Is it possible for you to try this on an another machine or with a clean version of VS Code?