在 Hugging Face Spaces 上设置 Phoenix 观察性仪表板以进行 LLM 应用程序追踪

作者：Andrew Reed

Phoenix 是由 Arize AI 开发的开源观察性库，旨在进行实验、评估和故障排除。它允许 AI 工程师和数据科学家快速可视化他们的数据、评估性能、追踪问题，并导出数据以进行改进。

追踪

数据集

实验

在本 Notebook 中，我们将展示如何在 Hugging Face Spaces 上部署 Phoenix 观察性仪表板，并配置它以自动追踪 LLM 调用，从而全面了解 LLM 应用程序的内部工作原理。

第一步：在 Hugging Face Spaces 上部署 Phoenix

虽然 Phoenix 提供了一个用于本地开发的 Notebook 高级选项，它也可以通过 Docker 作为独立仪表板部署。长时间运行的托管仪表板是提供 LLM 应用行为集中视图的好方法，并且有助于团队协作。Hugging Face Spaces 提供了一种简单的方式来托管 ML 应用，并支持可选的持久存储，且它对自定义 Docker 镜像的支持使其成为托管 Phoenix 的理想平台——我们来看看它是如何工作的！

首先，我们将 复制示例空间

我们可以将空间配置为私有或公开，并且它可以位于我们的用户命名空间或组织命名空间中。我们可以使用默认的免费 CPU 配置，并且重要的是，我们可以指定要为该空间附加一个持久磁盘。

点击“复制空间”后，Docker 镜像将开始构建。这将需要几分钟时间完成，之后我们将看到一个空的 Phoenix 仪表板。

第二步：配置应用追踪

现在我们已经有了一个运行中的 Phoenix 仪表板，可以使用 OpenTelemetry TracerProvider 配置应用程序，以自动追踪 LLM 调用。在这个例子中，我们将使用 OpenAI 客户端库来给应用程序加上监控，并追踪从 openai Python 包发出的 LLM 调用，这些调用会被发送到 Hugging Face 的 Serverless 推理 API 上运行的 LLM。

首先，我们需要安装必要的库：

!pip install -q arize-phoenix arize-phoenix-otel openinference-instrumentation-openai openai huggingface-hub

接下来，我们将使用 huggingface_hub 库登录到 Hugging Face。这将允许我们生成所需的身份验证，以便为我们的 Space 和 Serverless 推理 API 提供认证。请确保用于身份验证的 Hugging Face 令牌具有正确的权限，以便访问存放你 Space 的组织。

from huggingface_hub import interpreter_login

interpreter_login()

现在，我们可以将 Phoenix 客户端配置 为连接到我们运行中的 Phoenix 仪表板：

1. 注册 Phoenix 追踪提供者：
   • 指定我们选择的 project_name
   • 将 endpoint 设置为我们 Space 的主机名（可以通过仪表板 UI 中的“设置”标签找到，如下图所示）
   • 将 headers 设置为访问 Space 所需的 Hugging Face 头信息
2. 在我们的应用代码中，使用 OpenAI 追踪提供者进行监控

>>> from phoenix.otel import register
>>> from huggingface_hub.utils import build_hf_headers
>>> from openinference.instrumentation.openai import OpenAIInstrumentor

>>> # 1. Register the Phoenix tracer provider
>>> tracer_provider = register(
...     project_name="test",
...     endpoint="https://andrewrreed-phoenix-arize-observability-demo.hf.space" + "/v1/traces",
...     headers=build_hf_headers(),
... )

>>> # 2. Instrument our application code to use the OpenAI tracer provider
>>> OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

🔭 OpenTelemetry Tracing Details 🔭
|  Phoenix Project: test
|  Span Processor: SimpleSpanProcessor
|  Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces
|  Transport: HTTP
|  Transport Headers: {'user-agent': '****', 'authorization': '****'}
|  
|  Using a default SpanProcessor. `add_span_processor` will overwrite this default.
|  
|  `register` has set this TracerProvider as the global OpenTelemetry default.
|  To disable this behavior, call `register` with `set_global_tracer_provider=False`.

第三步：发起调用并在 Phoenix 仪表板中查看追踪

现在，我们可以调用 LLM 并在 Phoenix 仪表板中查看追踪。我们使用 OpenAI 客户端向 Hugging Face 的 Serverless 推理 API 发起调用，该 API 已经配置为与 Phoenix 配合工作。在这个例子中，我们使用的是 meta-llama/Llama-3.1-8B-Instruct 模型。

>>> from openai import OpenAI
>>> from huggingface_hub import get_token

>>> client = OpenAI(
...     base_url="https://api-inference.huggingface.co/v1/",
...     api_key=get_token(),
... )

>>> messages = [{"role": "user", "content": "What does a llama do for fun?"}]

>>> response = client.chat.completions.create(
...     model="meta-llama/Llama-3.1-8B-Instruct",
...     messages=messages,
...     max_tokens=500,
... )

>>> print(response.choices[0].message.content)

Llamas are intelligent and social animals, and they do have ways to entertain themselves and have fun. While we can't directly ask a llama about its personal preferences, we can observe their natural behaviors and make some educated guesses. Here are some things that might bring a llama joy and excitement:

1. **Socializing**: Llamas are herd animals and they love to interact with each other. They'll often engage in play-fighting, neck-wrestling, and other activities to establish dominance and strengthen social bonds. When llamas have a strong social network, it can make them feel happy and content.
2. **Exploring new environments**: Llamas are naturally curious creatures, and they love to explore new surroundings. They'll often investigate their surroundings, sniffing and investigating new sights, sounds, and smells.
3. **Playing with toys**: While llamas don't need expensive toys, they do enjoy playing with objects that stimulate their natural behaviors. For example, a ball or a toy that mimics a target can be an entertaining way to engage them.
4. **Climbing and jumping**: Llamas are agile and athletic animals, and they enjoy using their limbs to climb and jump over obstacles. Providing a safe and stable area for them to exercise their physical abilities can be a fun and engaging experience.
5. **Browsing and foraging**: Llamas have a natural instinct to graze and browse, and they enjoy searching for tasty plants and shrubs. Providing a variety of plants to munch on can keep them engaged and entertained.
6. **Mentally stimulating activities**: Llamas are intelligent animals, and they can benefit from mentally stimulating activities like problem-solving puzzles or learning new behaviors (like agility training or obedience training).

Some fun activities you can try with a llama include:

* Setting up an obstacle course or agility challenge
* Creating a "scavenger hunt" with treats and toys
* Introducing new toys or objects to stimulate their curiosity
* Providing a variety of plants and shrubs to browse and graze on
* Engaging in interactive games like "follow the leader" or "find the treat"

Remember to always prioritize the llama's safety and well-being, and to consult with a veterinarian or a trained llama handler before attempting any new activities or introducing new toys.

如果我们返回到 Phoenix 仪表板，我们可以看到从 LLM 调用捕获并显示的追踪信息！如果你在配置 Space 时使用了持久磁盘，那么每次重启 Space 时，所有的追踪信息都会被保存。

奖励部分：使用 CrewAI 追踪多代理应用程序

观察性的真正优势在于能够追踪和检查复杂的 LLM 工作流。在这个例子中，我们将安装并使用 CrewAI 来追踪一个多智能体应用程序。

!pip install -q openinference-instrumentation-crewai crewai crewai-tools

和之前一样，我们将注册 Phoenix 追踪提供者并给应用代码加上监控，但这次我们还需要取消现有 OpenAI 追踪提供者的监控，以避免冲突。

>>> from opentelemetry import trace
>>> from openinference.instrumentation.crewai import CrewAIInstrumentor

>>> # 0. Uninstrument existing tracer provider and clear the global tracer provider
>>> OpenAIInstrumentor().uninstrument()
>>> if trace.get_tracer_provider():
...     trace.get_tracer_provider().shutdown()
...     trace._TRACER_PROVIDER = None  # Reset the global tracer provider

>>> # 1. Register the Phoenix tracer provider
>>> tracer_provider = register(
...     project_name="crewai",
...     endpoint="https://andrewrreed-phoenix-arize-observability-demo.hf.space" + "/v1/traces",
...     headers=build_hf_headers(),
... )

>>> # 2. Instrument our application code to use the OpenAI tracer provider
>>> CrewAIInstrumentor().instrument(tracer_provider=tracer_provider)

🔭 OpenTelemetry Tracing Details 🔭
|  Phoenix Project: crewai
|  Span Processor: SimpleSpanProcessor
|  Collector Endpoint: https://andrewrreed-phoenix-arize-observability-demo.hf.space/v1/traces
|  Transport: HTTP
|  Transport Headers: {'user-agent': '****', 'authorization': '****'}
|  
|  Using a default SpanProcessor. `add_span_processor` will overwrite this default.
|  
|  `register` has set this TracerProvider as the global OpenTelemetry default.
|  To disable this behavior, call `register` with `set_global_tracer_provider=False`.

现在，我们将使用 CrewAI 定义一个多智能体应用程序，研究并撰写一篇关于观察性和追踪在 LLM 应用中的重要性的博客。

>>> import os
>>> from huggingface_hub import get_token
>>> from crewai_tools import SerperDevTool
>>> from crewai import LLM, Agent, Task, Crew, Process

>>> # Define our LLM using HF's Serverless Inference API
>>> llm = LLM(
...     model="huggingface/meta-llama/Llama-3.1-8B-Instruct",
...     api_key=get_token(),
...     max_tokens=1024,
... )

>>> # Define a tool for searching the web
>>> os.environ["SERPER_API_KEY"] = "YOUR_SERPER_API_KEY"  # must set this value in your environment
>>> search_tool = SerperDevTool()

>>> # Define your agents with roles and goals
>>> researcher = Agent(
...     role="Researcher",
...     goal="Conduct thorough research on up to date trends around a given topic.",
...     backstory="""You work at a leading tech think tank. You have a knack for dissecting complex data and presenting actionable insights.""",
...     verbose=True,
...     allow_delegation=False,
...     tools=[search_tool],
...     llm=llm,
...     max_iter=1,
... )
... writer = Agent(
...     role="Technical Writer",
...     goal="Craft compelling content on a given topic.",
...     backstory="""You are a technical writer with a knack for crafting engaging and informative content.""",
...     llm=llm,
...     verbose=True,
...     allow_delegation=False,
...     max_iter=1,
... )

>>> # Create tasks for your agents
>>> task1 = Task(
...     description="""Conduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.
...   Identify key trends, breakthrough technologies, and potential industry impacts.""",
...     expected_output="Full analysis report in bullet points",
...     agent=researcher,
... )

>>> task2 = Task(
...     description="""Using the insights provided, develop an engaging blog
...   post that highlights the importance of observability and tracing in LLM applications.
...   Your post should be informative yet accessible, catering to a tech-savvy audience.
...   Make it sound cool, avoid complex words so it doesn't sound like AI.""",
...     expected_output="Blog post of at least 3 paragraphs",
...     agent=writer,
... )

>>> # Instantiate your crew with a sequential process
>>> crew = Crew(
...     agents=[researcher, writer],
...     tasks=[task1, task2],
...     verbose=True,
...     process=Process.sequential,
... )

>>> # Get your crew to work!
>>> result = crew.kickoff()

>>> print("------------ FINAL RESULT ------------")
>>> print(result)

# Agent: Researcher
## Task: Conduct comprehensive research and analysis of the importance of observability and tracing in LLM applications.
  Identify key trends, breakthrough technologies, and potential industry impacts.


# Agent: Researcher
## Using tool: Search the internet
## Tool Input: 
"{\"search_query\": \"importance of observability and tracing in LLM applications\"}"
## Tool Output: 

Search results: Title: LLM Observability: The 5 Key Pillars for Monitoring Large Language ...
Link: https://arize.com/blog-course/large-language-model-monitoring-observability/
Snippet: Why leveraging the five pillars of LLM observability is essential for ensuring performance, reliability, and seamless LLM applications.
---
Title: Observability of LLM Applications: Exploration and Practice from the ...
Link: https://www.alibabacloud.com/blog/observability-of-llm-applications-exploration-and-practice-from-the-perspective-of-trace_601604
Snippet: This article clarifies the technical challenges of observability by analyzing LLM application patterns and different concerns.
---
Title: What is LLM Observability? - The Ultimate LLM Monitoring Guide
Link: https://www.confident-ai.com/blog/what-is-llm-observability-the-ultimate-llm-monitoring-guide
Snippet: Observability tools collect and correlate logs, real-time evaluation metrics, and traces to understand the context of unexpected outputs or ...
---
Title: An Introduction to Observability for LLM-based applications using ...
Link: https://opentelemetry.io/blog/2024/llm-observability/
Snippet: Why Observability Matters for LLM Applications · It's vital to keep track of how often LLMs are being used for usage and cost tracking. · Latency ...
---
Title: Understanding LLM Observability - Key Insights, Best Practices ...
Link: https://signoz.io/blog/llm-observability/
Snippet: LLM Observability is essential for maintaining reliable, accurate, and efficient AI applications. Focus on the five pillars: evaluation, tracing ...
---
Title: LLM Observability Tools: 2024 Comparison - lakeFS
Link: https://lakefs.io/blog/llm-observability-tools/
Snippet: LLM observability is the process that enables monitoring by providing full visibility and tracing in an LLM application system, as well as newer ...
---
Title: From Concept to Production with Observability in LLM Applications
Link: https://hadijaveed.me/2024/03/05/tracing-and-observability-in-llm-applications/
Snippet: Traces are essential to understanding the full “path” a request takes in your application, e.g, prompt, query-expansion, RAG retrieved top-k ...
---
Title: The Importance of LLM Observability: A Technical Deep Dive
Link: https://www.linkedin.com/pulse/importance-llm-observability-technical-deep-dive-patrick-carroll-trlqe
Snippet: LLM observability is crucial for any technical team that wants to maintain and improve the reliability, security, and performance of their AI- ...
---
Title: Observability and Monitoring of LLMs - TheBlue.ai
Link: https://theblue.ai/blog/llm-observability-en/
Snippet: LLM-Observability is crucial to maximize the performance and reliability of Large Language Models (LLMs). By systematically capturing and ...
---


返回 Phoenix 仪表板后，我们可以在新项目 “crewai” 中看到来自我们多智能体应用程序的追踪信息！