Free Databricks Certified Generative AI Engineer Associate Exam Databricks-Generative-AI-Engineer-Associate Exam Practice Test

A Generative AI Engineer is designing a chatbot for a gaming company that aims to engage users on its platform while its users play online video games.Which metric would help them increase user engagement and retention for their platform?

• Randomness
• Diversity of responses
• Lack of relevance
• Repetition of responses

A Generative AI Engineer is designing an LLM-powered live sports commentary platform. The platform provides real-time updates and LLM-generated analyses for any users who would like to have live summaries, rather than reading a series of potentially outdated news articles.Which tool below will give the platform access to real-time data for generating game analyses based on the latest game scores?

• DatabrickslQ
• Foundation Model APIs
• Feature Serving
• AutoML

A Generative Al Engineer interfaces with an LLM with prompt/response behavior that has been trained on customer calls inquiring about product availability. The LLM is designed to output “In Stock” if the product is available or only the term “Out of Stock” if not.Which prompt will work to allow the engineer to respond to call classification labels correctly?

• Respond with “In Stock” if the customer asks for a product.
• You will be given a customer call transcript where the customer asks about product availability. The outputs are either “In Stock” or “Out of Stock”. Format the output in JSON, for example:{“call_id”: “123”, “label”: “In Stock”}.
• Respond with “Out of Stock” if the customer asks for a product.
• You will be given a customer call transcript where the customer inquires about product availability. Respond with “In Stock” if the product is available or “Out of Stock” if not.

A Generative AI Engineer has created a RAG application which can help employees retrieve ans.wers from an internal knowledge base, such as Confluence pages or Google Drive. The prototype application is now working with some positive feedback from internal company testers. Now the Generative Al Engineer wants to formally evaluate the system’s performance and understand where to focus their efforts to further improve the system.How should the Generative AI Engineer evaluate the system?

• Use cosine similarity score to comprehensively evaluate the quality of the final generated answ.ers.
• Curate a dataset that can test the retrieval and generation components of the system separately. Use MLflow’s built in evaluation metrics to perform the evaluation on the retrieval and generation components.
• Benchmark multiple LLMs with the same data and pick the best LLM for the job.
• Use an LLM-as-a-judge to evaluate the quality of the final ans.wers generated.

A Generative Al Engineer is responsible for developing a chatbot to enable their company’s internal HelpDesk Call Center team to more quickly find related tickets and provide resolution. While creating the GenAI application work breakdown tasks for this project, they realize they need to start planning which data sources (either Unity Catalog volume or Delta table) they could choose for this application. They have collected several candidate data sources for consideration:call_rep_history: a Delta table with primary keys representative_id, call_id. This table is maintained to calculate representatives’ call resolution from fields call_duration and call start_time.transcript Volume: a Unity Catalog Volume of all recordings as a *.wav files, but also a text transcript as *.txt files.call_cust_history: a Delta table with primary keys customer_id, cal1_id. This table is maintained to calculate how much internal customers use the HelpDesk to make sure that the charge back model is consistent with actual service use.call_detail: a Delta table that includes a snapshot of all call details updated hourly. It includes root_cause and resolution fields, but those fields may be empty for calls that are still active.maintenance_schedule – a Delta table that includes a listing of both HelpDesk application outages as well as planned upcoming maintenance downtimes.They need sources that could add context to best identify ticket root cause and resolution. Which TWO sources do that? (Choose two.)

• call_cust_history
• maintenance_schedule
• call_rep_history
• call_detail
• transcript Volume

Generative AI Engineer at an electronics company just deployed a RAG application for customers to ask questions about products that the company carries. However, they received feedback that the RAG response often returns information about an irrelevant product.What can the engineer do to improve the relevance of the RAG’s response?

• Assess the quality of the retrieved context
• Implement caching for frequently asked questions
• Use a different LLM to improve the generated response
• Use a different semantic similarity search algorithm

A team wants to serve a code generation model as an assistant for their software developers. It should support multiple programming languages. Quality is the primary objective.Which of the Databricks Foundation Model APIs, or models available in the Marketplace, would be the best fit?

• Llama2-70b
• BGE-large
• MPT-7b
• CodeLlama-34B

A Generative AI Engineer is designing a RAG application for answe.ring user questions on technical regulations as they learn a new sport.What are the steps needed to build this RAG application and deploy it?

• Ingest documents from a source –> Index the documents and saves to Vector Search –> User submits queries against an LLM –> LLM retrieves relevant documents –> Evaluate model –> LLM generates a response –> Deploy it using Model Serving
• Ingest documents from a source –> Index the documents and save to Vector Search –> User submits queries against an LLM –> LLM retrieves relevant documents –> LLM generates a response -> Evaluate model –> Deploy it using Model Serving
• Ingest documents from a source –> Index the documents and save to Vector Search –> Evaluate model –> Deploy it using Model Serving
• User submits queries against an LLM –> Ingest documents from a source –> Index the documents and save to Vector Search –> LLM retrieves relevant documents –> LLM generates a response –> Evaluate model –> Deploy it using Model Serving

A Generative Al Engineer is tasked with developing a RAG application that will help a small internal group of experts at their company ans.wer specific questions, augmented by an internal knowledge base. They want the best possible quality in the options, and neither latency nor throughput is a huge concern given that the user group is small and they’re willing to wait for the best option. The topics are sensitive in nature and the data is highly confidential and so, due to regulatory requirements, none of the information is allowed to be transmitted to third parties.Which model meets all the Generative Al Engineer’s needs in this situation?

• Dolly 1.5B
• OpenAI GPT-4
• BGE-large
• Llama2-70B

A Generative AI Engineer just deployed an LLM application at a digital marketing company that assists with answ.ering customer service inquiries.Which metric should they monitor for their customer service LLM application in production?

• Number of customer inquiries processed per unit of time
• Energy usage per query
• Final perplexity scores for the training of the model
• HuggingFace Leaderboard values for the base LLM