Skip to content

Update /generate.py to handle "thinking" models #1323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

srdas
Copy link
Collaborator

@srdas srdas commented Apr 14, 2025

Fixes Issue #1315

In models that return an entire chain of thought, /generate fails as the notebook file name becomes excessively long, as noted in the issue cited above.

The title returned contains all the thinking about the title, including proposed title text buried within qoutes inside the chain of thought (CoT). This modification extracts the suggested title in the CoT and uses it for the title.

The modification also adds a generic failsafe title in case the CoT fails to return a title. For non-CoT LLMs, the code does not pursue this special handling.

To test:
[1] Set chat model provider to Ollama and the model to huggingface.co/lmstudio-community/DeepCoder-14B-Preview-GGUF:Q6_K
[2] Try the following chat command: /generate write "hi" in the file "allo.txt" in python (this will take some time to run, as the model works through its process of reflection).

As shown:
Screenshot 2025-04-14 at 2 52 29 PM
Result:
Screenshot 2025-04-14 at 2 52 41 PM
Screenshot 2025-04-14 at 2 53 23 PM

Additional tests undertaken:

  • The chat command /generate add two numbers in python with the same LLM gives the notebook title: Adding Two Numbers in Python - Basic to Advanced.ipynb.
  • The chat command /generate write "hi" in the file "allo.txt" in python generates a title: Write 'hi' to allo.txt.ipynb. (Here the LLM is deepseek-chat with provider OpenRouter.)

@srdas srdas added the bug Bugs reported by users label Apr 14, 2025
@srdas srdas requested a review from dlqqq April 14, 2025 23:31
dlqqq and others added 3 commits April 15, 2025 14:07
* add test catching error with empty touched config.json

* handle empty touched config.json, passes test

* ensure schema file doesn't have duplicated types

* pre-commit
@srdas srdas requested a review from JGuinegagne April 15, 2025 21:11
@srdas srdas requested a review from JGuinegagne April 16, 2025 18:01
@srdas srdas requested review from dlqqq and removed request for dlqqq April 16, 2025 18:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bugs reported by users
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants