how to have longer output using gpt-3.5-turbo-16k model #28
Labels
No labels
bug
dependencies
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
waiting-reply
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: bobmaster/matrix_chatgpt_bot#28
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
I manually changed the model in settings.js in node-chatgpt-api container to gpt-3.5-turbo-16k model and also specified max_token to say 12000 tokens. See below. However, when I asked the bot to create a 5000 words story, each time the generated text is quite short, maybe around 700 words. Is there elsewhere limiting the output length?
Enable debug mode and get the payload/response from console
217c189cbb/settings.example.js (L37)
not showing anything about the payload/reponse from console after enabling debug mode. wierd. I am using your forked docker container at https://github.com/hibobmaster/node-chatgpt-api. relevant?
Jun 24 10:24:22 arm1 -16k[2686543]: bot-chatgpt | 2023-06-24 02:24:22,759 - INFO - prompt: write a 4000 words story about a boy likes fishing
Jun 24 10:24:40 arm1 -16k[2686543]: bot-chatgpt | 2023-06-24 02:24:40,786 - INFO - Message received in room chatgpt-16k
Jun 24 10:24:40 arm1 -16k[2686543]: bot-chatgpt | ChatGPT-16K| Once upon a time, in a small coastal town called Seaview, there lived a young boy named Daniel. Daniel had always been fascinated by the sea and all its wonders, but his true passion lay in fishing. From an early age, he would spend hours on end with his father at the local pier, casting lines into the deep blue, hoping for a bite.
Jun 24 10:24:40 arm1 -16k[2686543]: bot-chatgpt |
Jun 24 10:24:40 arm1 -16k[2686543]: bot-chatgpt | Daniel's love for fishing was not just about catching fish; it went much deeper than that. He found solace in the rhythmic motion of casting the line, the gentle lapping of the waves against the shore, and the peacefulness that enveloped him as he waited patiently for a tug on his line. Fishing became his escape from the chaos of everyday life.
Jun 24 10:24:40 arm1 -16k[2686543]: bot-chatgpt |
|_
Sorry my fault. I was looking at the log info of matrix_chat_bot, not node-chatgpt-bot.
Got the log as below. It appears setting max_tokens correctly to 12000. why can't it produce 4000 words as requested? An issue of openai itself?
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api |
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | https://api.openai.com/v1/chat/completions
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | {
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | model: 'gpt-3.5-turbo-16k-0613',
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | temperature: 0.8,
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | max_tokens: 12000,
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | top_p: 1,
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | presence_penalty: 1,
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | stop: [ '||>', '\nUser:', '<|diff_marker|>' ],
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | messages: [
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | {
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | role: 'system',
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | name: 'instructions',
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | content: '||>Instructions:\n' +
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | 'You are ChatGPT, a large language model trained by OpenAI. Respond conversationally.\n' +
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | 'Current date: June 24, 2023\n' +
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | '\n'
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | },
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | {
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | role: 'system',
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | content: '||>User:\n' +
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | 'write a 4000 words story about a boy likes fishing\n' +
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | '||>ChatGPT:\n'
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | }
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | ]
Jun 24 13:53:47 arm1 node_chatgpt_api[2740547]: node_chatgpt_api | }
I use another tool and got the same result:
prompt: write a 4000 words story about a boy likes fishing
Maybe you could try openai playground to verify it.
https://platform.openai.com/playground?mode=chat&model=gpt-3.5-turbo-16k
wierd. On playground, the maximum token is only 2048 even for gpt-3.5-turbo-16k model.
Some other people have the same question.
https://www.reddit.com/r/ChatGPT/comments/10ddllh/got_tips_for_how_to_get_chatgpt_to_write_a_long/