Exploring Advanced Prompt Engineering with Google’s New Gemma Models

Today, Google launched a new set of models named Gemma. These models are based on the same tech and research used for creating the Gemini models. Gemma comes in two versions: Gemma 2B, which is small enough for the free tier of Google Colab, and Gemma 7B.

This launch is significant because, unlike what other companies have released up until now, it claims to closely parallel a leading Large Language Model that is not open to the public.

Therefore, we may be able to employ advanced prompt engineering techniques on Gemma, which are not possible without an open AI model, and subsequently apply them on Gemini, a closed AI model.

For instance, we could select candidate prompts with low perplexities on Gemma, and then use these prompts on Gemini, where we are not allowed to calculate the perplexity of each prompt. If both models are indeed similar, this approach could actually improve our prompts for Gemini, even though we don’t have access to the model weights.

I am looking forward to seeing studies that experiment with this approach. If Gemma is as similar to Gemini as they say, we might see some great results.

Gemma: Introducing new state-of-the-art open models
https://blog.google/technology/developers/gemma-open-models/

Demystifying Prompts in Language Models via Perplexity Estimation https://arxiv.org/abs/2212.04037 “Over a wide range of tasks, we show that the lower the perplexity of the prompt is, the better the prompt is able to perform the task.”

Gemma on Hugging Face

https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b