October 6, 2025briolson69

LLM Prompt & Evaluative Rubric Writing for Hospitality Company

This project demonstrates my ability to design structured evaluation workflows for large language models (LLMs).

It includes a prompt, scaffolding, and a rubric—the same three-part framework used in professional AI evaluation environments (e.g., Mercor, Scale, or DataAnnotation).

The rubric shown below is a set of precise, reproducible criteria used by human evaluators to assess the quality, factual grounding, tone, and structure of AI-generated responses. Each criterion ties directly to the task instructions and references specific source documents to ensure consistent, objective scoring across evaluators.

The goal: to help align LLM-generated content with client expectations in marketing, content writing, and conversational AI domains.

Sources

  • Source 1 – The M Apartments Handbook (PDF)
  • Source 2 – Where to Stay in Spokane (PDF)
  • Source 3 – Stay Alfred at The M Apartments – Spokane (PDF)
  • Source 4 – What to Do, Where to Stay in Spokane (PDF)

Prompt (Research/Write Style)

Write a 500–700-word blog post covering the building, neighborhood, and city highlights for The M Apartments (Stay Alfred at The M) in downtown Spokane, WA. Blend practical information for guests with touristic elements such as entertainment and dining.

Use information from the provided PDFs in this order of priority:

  1. The M Handbook
  2. Where to Stay in Spokane
  3. Stay Alfred at The M
  4. What to Do, Where to Stay in Spokane

The post should:

  • Identify the apartment features and building’s key amenities (rooftop lounge, gym, in-unit kitchens, private bedrooms, washer/dryer). (Source 1)
  • Highlight nearby attractions, parks, theaters, trails, festivals, or markets. (Source 4)
  • Feature 2–3 popular venues within 1 mile (parks, breweries, music venues), summarizing signature items or atmosphere. (Source 3)
  • Include hyperlocal details: walking directions and block/minute estimates to the closest ATM, coffee shop, grocery store, and pharmacy. (Source 3)
  • Follow a warm, second-person voice (“you”), friendly and visitor-focused. Brand voice: quirky and casual — make guests feel like locals. (Source 2)
  • Use bullet points or subheads for clarity, e.g.: “Location & Amenities,” “Downtown Attractions,” “Where to Eat Nearby,” “Things to Do.”

Scaffolding

  1. List rooftop lounge and gym; describe benefits to guests.
  2. Describe in-unit kitchens, private bedrooms, and washer/dryer.
  3. Identify nearby parks, trails, theaters, festivals, and markets; include block/minute estimates if available.
  4. Feature one popular venue within 1 mile; include signature items, atmosphere, walking directions.
  5. Feature second (and optional third) venue with same details.
  6. List nearest ATMs, coffee shops, groceries, and pharmacies with walking/block estimates.
  7. Write in warm, quirky, casual, second-person voice; playful/insider tips optional.
  8. Use subheads and bullet points for clarity; organize logically.
  9. Proofread for grammar, spelling, and sentence flow.
  10. Ensure the post alternates between practical info and attractions for balance.

Rubric

This rubric is designed for use by human AI evaluators reviewing model-generated responses to the prompt below. Each criterion specifies what an ideal output should include and identifies which source documents should be referenced when determining accuracy, completeness, and alignment with the task. Together, these criteria ensure consistent, objective, and replicable evaluations across annotators.

Rubric (10 Criteria)

Criterion 1: Amenities — rooftop lounge and gym

Sources: The M Handbook (Source 1)
Justification: Accurately report rooftop lounge and gym; describe practical benefits for guests.
Weight: Primary objective
Criterion Type: Quality/Accuracy
Dependent Criteria: none

Criterion 2: Amenities — in-unit kitchens, private bedrooms, washer/dryer

Sources: The M Handbook (Source 1)
Justification: Accurate reporting ensures readers understand convenience features.
Weight: Primary objective
Criterion Type: Quality/Accuracy
Dependent Criteria: 1

Criterion 3: Nearby attractions — parks, trails, theaters, festivals, markets

Sources: What to Do, Where to Stay in Spokane (Source 4)
Justification: Identify attractions and include walking/driving times if available.
Weight: Primary objective
Criterion Type: Quality/Accuracy
Dependent Criteria: none

Criterion 4: Venue 1 — popular venue with signature items and directions

Sources: Stay Alfred at The M (Source 3)
Justification: Feature one venue accurately, including atmosphere and hyperlocal directions.
Weight: Primary objective
Criterion Type: Quality/Accuracy
Dependent Criteria: 3

Criterion 5: Venue 2 (and optional 3) — popular venue with signature items and directions

Sources: Stay Alfred at The M (Source 3)
Justification: Feature a second venue accurately, with signature items and directions.
Weight: Primary objective
Criterion Type: Quality/Accuracy
Dependent Criteria: 4

Criterion 6: Hyperlocal practical information — ATMs, coffee shops, groceries, pharmacy

Sources: Stay Alfred at The M (Source 3)
Justification: Provide accurate walking/block estimates to these locations for practical use.
Weight: Primary objective
Criterion Type: Quality/Accuracy
Dependent Criteria: 5

Criterion 7: Tone/voice — warm, quirky, casual, second-person

Sources: Where to Stay in Spokane (Source 2)
Justification: Brand voice should make readers feel like locals; playful/insider tips optional.
Weight: Not primary objective
Criterion Type: Quality/Style
Dependent Criteria: none

Criterion 8: Organization — clear subheads and bullet points

Sources: Prompt instructions
Justification: Subheads like “Location & Amenities,” “Downtown Attractions,” “Where to Eat Nearby,” and “Tips for Fitting In” improve readability and clarity.
Weight: Not primary objective
Criterion Type: Quality/Style
Dependent Criteria: 7

Criterion 9: Grammar, spelling, and sentence flow

Sources: Prompt
Justification: Proper grammar, no typos, and smooth transitions ensure professional readability.
Weight: Not primary objective
Criterion Type: Quality/Style
Dependent Criteria: 7, 8

Criterion 10: Balance practical stay info vs tourist attractions

Sources: Sources 1–4
Justification: Alternate between amenities/practical info and nearby attractions/venues to give a full picture.
Weight: Primary objective
Criterion Type: Quality/Accuracy
Dependent Criteria: 1–6

✅ Author’s Note:
This project illustrates end-to-end prompt and rubric design for evaluating long-form, brand-aligned writing. It reflects my broader focus on human-in-the-loop content systems and conversational UX design.

Leave a comment