Text Generation with Google Gemini

Demo: Using Parameters

Now, you’ll experiment with the parameters. At the bottom of your Python script from the last demo, add a code cell and then add:

genai.get_model('models/gemini-1.5-pro')

This allows you to see the current settings on the Gemini 1.5 Pro model. Execute this cell and note the temperature, top_p, and the top_k values. These values control attributes of text generation. Add a new code cell and then add:

generation_config = genai.types.GenerationConfig(
  candidate_count = 1,
  # stop_sequences = [';']
  # max_output_tokens = 32000
  # temperature = 0.0,
  # top_k = 4,
  # top_p = 1
)

Comment these lines for now. In a new code cell, now add:

response = model.generate_content(
  'Write a rap about doing the dishes',
  generation_config = generation_config
)
print(response.text)

This code creates a generation_config object, and the different parameter values are enumerated in the comments. The model generates content from this prompt about doing the dishes and then stores it in the response. You then print the response in a text format.

Execute this code line by line to see what the default parameter values produce. Most likely, you’ll get several verses. The model may also give a few chorus verses. The model, with its creative rap, made a mundane task more fun.

Now, come up to the gen config object and uncomment this stop_sequences line. Change the semi-colon to a comma.

stop_sequences = [',']

Execute this code and the next line. Wait for the response to come up and see how it’s cut off at the first instance of a comma. Now, comment back this stop_sequences line and uncomment the next line with max_output_tokens. Change this value to 25.

max_output_tokens = 25

Now execute this line and the next line with response. The model now prints only the first 25 tokens. Now, place a hashtag in front of the max_output_tokens to comment it again.

Uncomment the temperature and change this value to 0.2. Now come down to response and change this prompt to Tell me a joke. Now execute these two code cells. The response most likely results in some well-known or safe jokes. Now change the temperature to 2 and execute this code again. Wait for the response to come up. Now it will tell a much more complicated joke that can be edgy or even something that does not make much sense.

Now, comment back the temperature, uncomment top_k and change the value of top_k to 1. Change the prompt to Write a poem about a cat. Execute these lines again. The output will produce the most likely next word at each step, potentially leading to a predictable and a cliché poem.

Now, change the value to 20 and execute this code again. This will generate a much more creative and unpredictable poem with turns of phrase and imagery.

Now comment top_k and uncomment top_p. Change the value to 0.5 and change the prompt to Generate a list of creative date ideas. Execute to see the response. You’ll see a value of top_p as 0.5 will suggest safer and more conventional date ideas. Now, change the top_p value to 0.95 and execute the code again. This time you can see the date ideas are a lot more unconventional and creative.

Lastly, uncomment and change the temperature to 0.6, top_k to 10, and top_p to 0.7. Now, change the generate_content prompt to Write a short story about a time-traveling detective. Execute the code and view the result. This combination will likely result in a coherent and engaging story with a good mix of familiar tropes and unexpected plot twists.

The optimal values will depend on the specific prompt and desired output. Experimenting with different combinations is key to discovering the full range of possibilities.

Now, add a last code cell and add the following code:

model = genai.GenerativeModel('gemini-pro', generation_config =
  generation_config)

Executing this statement will apply the configuration model widely, and then you don’t have to pass in the configuration to every prompt following. You may choose if you want the configuration to be prompt specific or script wide.

Great work! You’ve created your own Python script and learned how to generate text with the Gemini API.