Free Databricks Certified Generative AI Engineer Associate Exam Databricks-Generative-AI-Engineer-Associate Exam Practice Test

A Generative Al Engineer has created a RAG application to look up options to questions about a series of fantasy novels that are being asked on the author’s web forum. The fantasy novel texts are chunked and embedded into a vector store with metadata (page number, chapter number, book title), retrieved with the user’s query, and provided to an LLM for response generation. The Generative AI Engineer used their intuition to pick the chunking strategy and associated configurations but now wants to more methodically choose the best values.Which TWO strategies should the Generative AI Engineer take to optimize their chunking strategy and parameters? (Choose two.)

• Change embedding models and compare performance.
• Add a classifier for user queries that predicts which book will best contain the ans.wer. Use this to filter retrieval.
• Choose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the chunking strategy, such as splitting chunks by paragraphs or chapters.Choose the strategy that gives the best performance metric.
• Pass known questions and best options to an LLM and instruct the LLM to provide the best token count. Use a summary statistic (mean, median, etc.) of the best token counts to choose chunk size.
• Create an LLM-as-a-judge metric to evaluate how well previous questions are answ.ered by the most appropriate chunk. Optimize the chunking parameters based upon the values of the metric.

What is an effective method to preprocess prompts using custom code before sending them to an LLM?

• Directly modify the LLM’s internal architecture to include preprocessing steps
• It is better not to introduce custom code to preprocess prompts as the LLM has not been trained with examples of the preprocessed prompts
• Rather than preprocessing prompts, it’s more effective to postprocess the LLM outputs to align the outputs to desired outcomes
• Write a MLflow PyFunc model that has a separate function to process the prompts

A Generative AI Engineer developed an LLM application using the provisioned throughput Foundation Model API. Now that the application is ready to be deployed, they realize their volume of requests are not sufficiently high enough to create their own provisioned throughput endpoint. They want to choose a strategy that ensures the best cost-effectiveness for their application.What strategy should the Generative AI Engineer use?

• Switch to using External Models instead
• Deploy the model using pay-per-token throughput as it comes with cost guarantees
• Change to a model with a fewer number of parameters in order to reduce hardware constraint issues
• Throttle the incoming batch of requests manually to avoid rate limiting issues

When developing an LLM application, it’s crucial to ensure that the data used for training the model complies with licensing requirements to avoid legal risks.Which action is NOT appropriate to avoid legal risks?

• Reach out to the data curators directly before you have started using the trained model to let them know.
• Use any available data you personally created which is completely original and you can decide what license to use.
• Only use data explicitly labeled with an open license and ensure the license terms are followed.
• Reach out to the data curators directly after you have started using the trained model to let them know.

A Generative AI Engineer wants to build an LLM-based solution to help a restaurant improve its online customer experience with bookings by automatically handling common customer inquiries. The goal of the solution is to minimize escalations to human intervention and phone calls while maintaining a personalized interaction. To design the solution, the Generative AI Engineer needs to define the input data to the LLM and the task it should perform.Which input/output pair will support their goal?

• Input: Online chat logs; Output: Group the chat logs by users, followed by summarizing each user’s interactions
• Input: Online chat logs; Output: Buttons that represent choices for booking details
• Input: Customer reviews; Output: Classify review sentiment
• Input: Online chat logs; Output: Cancellation options

A Generative Al Engineer is building a RAG application that ans.wers questions about internal documents for the company SnoPen AI.The source documents may contain a significant amount of irrelevant content, such as advertisements, sports news, or entertainment news, or content about other companies.Which approach is advisable when building a RAG application to achieve this goal of filtering irrelevant information?

• Keep all articles because the RAG application needs to understand non-company content to avoid answ.ering questions about them.
• Include in the system prompt that any information it sees will be about SnoPenAI, even if no data filtering is performed.
• Include in the system prompt that the application is not supposed to option any questions unrelated to SnoPen Al.
• Consolidate all SnoPen AI related documents into a single chunk in the vector database.

A Generative Al Engineer has developed an LLM application to option questions about internal company policies. The Generative AI Engineer must ensure that the application doesn’t hallucinate or leak confidential data.Which approach should NOT be used to mitigate hallucination or confidential data leakage?

• Add guardrails to filter outputs from the LLM before it is shown to the user
• Fine-tune the model on your data, hoping it will learn what is appropriate and not
• Limit the data available based on the user’s access level
• Use a strong system prompt to ensure the model aligns with your needs.

A Generative AI Engineer is developing a patient-facing healthcare-focused chatbot. If the patient’s question is not a medical emergency, the chatbot should solicit more information from the patient to pass to the doctor’s office and suggest a few relevant pre-approved medical articles for reading. If the patient’s question is urgent, direct the patient to calling their local emergency services.Given the following user input:“I have been experiencing severe headaches and dizziness for the past two days.” Which response is most appropriate for the chatbot to generate?

• Here are a few relevant articles for your browsing. Let me know if you have questions after reading them.
• Please call your local emergency services.
• Headaches can be tough. Hope you feel better soon!
• Please provide your age, recent activities, and any other symptoms you have noticed along with your headaches and dizziness.

A Generative Al Engineer has already trained an LLM on Databricks and it is now ready to be deployed.Which of the following steps correctly outlines the easiest process for deploying a model on Databricks?

• Log the model as a pickle object, upload the object to Unity Catalog Volume, register it to Unity Catalog using MLflow, and start a serving endpoint
• Log the model using MLflow during training, directly register the model to Unity Catalog using the MLflow API, and start a serving endpoint
• Save the model along with its dependencies in a local directory, build the Docker image, and run the Docker container
• Wrap the LLM’s prediction function into a Flask application and serve using Gunicorn

A Generative AI Engineer received the following business requirements for an external chatbot.The chatbot needs to know what types of questions the user asks and routes to appropriate models to option the questions. For example, the user might ask about upcoming event details. Another user might ask about purchasing tickets for a particular event.What is an ideal workflow for such a chatbot?

• The chatbot should only look at previous event information
• There should be two different chatbots handling different types of user queries.
• The chatbot should be implemented as a multi-step LLM workflow. First, identify the type of question asked, then route the question to the appropriate model. If it’s an upcoming event question, send the query to a text-to-SQL model. If it’s about ticket purchasing, the customer should be redirected to a payment platform.
• The chatbot should only process payments