Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ on:

permissions:
contents: write
pull-requests: write

concurrency: ci-${{ github.ref }}

Expand All @@ -37,7 +36,7 @@ jobs:
runs-on: ubuntu-latest
env:
PR_PATH: pull/${{github.event.number}}
BASE_URL: https://burr.dagworks.io/pull/${{github.event.number}}
BASE_URL: https://burr.apache.org/pull/${{github.event.number}}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
Expand All @@ -49,7 +48,7 @@ jobs:
- name: Sphinx build
run: |
sphinx-build docs -b dirhtml _build
echo "burr.dagworks.io" > _build/CNAME # keep the cname file which this clobbers -- todo, unhardcode
echo "burr.apache.org" > _build/CNAME # keep the cname file which this clobbers -- todo, unhardcode
- name: Comment on PR
uses: hasura/comment-progress@v2.2.0
if: github.ref != 'refs/heads/main'
Expand Down
4 changes: 1 addition & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,9 +160,7 @@ While Apache Burr is attempting something (somewhat) unique, there are a variety
## 🌯 Why the name Burr?

Apache Burr is named after [Aaron Burr](https://en.wikipedia.org/wiki/Aaron_Burr), founding father, third VP of the United States, and murderer/arch-nemesis of [Alexander Hamilton](https://en.wikipedia.org/wiki/Alexander_Hamilton).
What's the connection with Hamilton? This is [DAGWorks](www.dagworks.io)' second open-source library release after the [Apache Hamilton library](https://github.com/apache/hamilton)
We imagine a world in which Burr and Hamilton lived in harmony and saw through their differences to better the union. We originally
built Apache Burr as a _harness_ to handle state between executions of Apache Hamilton DAGs (because DAGs don't have cycles),
What's the connection with (Apache) Hamilton? We imagine a world in which Burr and Hamilton lived in harmony and saw through their differences to better the union. Originally Apache Burr was built as a _harness_ to handle state between executions of Apache Hamilton DAGs (because DAGs don't have cycles),
but realized that it has a wide array of applications and decided to release it more broadly.

# Testimonials
Expand Down
8 changes: 5 additions & 3 deletions burr/core/application.py
Original file line number Diff line number Diff line change
Expand Up @@ -201,9 +201,9 @@ def _state_update(state_to_modify: State, modified_state: State) -> State:

This is suboptimal -- we should not be observing the state, we should be using the state commands and layering in deltas.
That said, we currently eagerly evaluate the state at all operations, which means we have to do it this way. See
https://github.com/DAGWorks-Inc/burr/issues/33 for a more detailed plan.
https://github.com/apache/burr/issues/33 for a more detailed plan.

This function was written to solve this issue: https://github.com/DAGWorks-Inc/burr/issues/28.
This function was written to solve this issue: https://github.com/apache/burr/issues/28.


:param state_subset_pre_update: The subset of state passed to the update() function
Expand Down Expand Up @@ -851,6 +851,8 @@ def __init__(
spawning_parent_pointer=spawning_parent_pointer,
)

# @telemetry.capture_function_usage # todo -- capture usage when we break this up into one that isn't called internally
# This will be doable when we move sequence ID to the beginning of the function https://github.com/apache/burr/pull/73
@_call_execute_method_pre_post(ExecuteMethod.step)
def step(self, inputs: Optional[Dict[str, Any]] = None) -> Optional[Tuple[Action, dict, State]]:
"""Performs a single step, advancing the state machine along.
Expand Down Expand Up @@ -2257,7 +2259,7 @@ def with_graph(self, graph: Graph) -> "ApplicationBuilder[StateType]":

def with_parallel_executor(self, executor_factory: lambda: Executor):
"""Assigns a default executor to be used for recursive/parallel sub-actions. This effectively allows
for executing multiple Burr apps in parallel. See https://burr.dagworks.io/concepts/parallelism/
for executing multiple Burr apps in parallel. See https://burr.apache.org/concepts/parallelism/
for more details.

This will default to a simple threadpool executor, meaning that you will be bound by the number of threads
Expand Down
2 changes: 1 addition & 1 deletion burr/core/state.py
Original file line number Diff line number Diff line change
Expand Up @@ -307,7 +307,7 @@ def apply_operation(self, operation: StateDelta) -> "State[StateType]":
# This ensures we only copy the fields that are read by value
# and copy the others by value
# TODO -- make this more efficient when we have immutable transactions
# with event-based history: https://github.com/DAGWorks-Inc/burr/issues/33
# with event-based history: https://github.com/apache/burr/issues/33
if field in new_state:
# currently the reads() includes optional fields
# We should clean that up, but this is an internal API so not worried now
Expand Down
2 changes: 1 addition & 1 deletion burr/integrations/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,6 @@ def require_plugin(import_error: ImportError, plugin_name: str):
raise ImportError(
f"Missing plugin {plugin_name}! To use the {plugin_name} plugin, you must install the 'extras' target [{plugin_name}] with burr[{plugin_name}] "
f"(replace with your package manager of choice). Note that, if you're using poetry, you cannot install burr with burr[start], so "
f"you'll have to install the components individually. See https://burr.dagworks.io/getting_started/install/ "
f"you'll have to install the components individually. See https://burr.apache.org/getting_started/install/ "
f"for more details."
) from import_error
8 changes: 4 additions & 4 deletions burr/tracking/server/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -257,10 +257,10 @@ def safe_json_load(line: bytes):

def get_uri(project_id: str) -> str:
project_id_map = {
"demo_counter": "https://github.com/DAGWorks-Inc/burr/tree/main/examples/hello-world-counter",
"demo_tracing": "https://github.com/DAGWorks-Inc/burr/tree/main/examples/tracing-and-spans/application.py",
"demo_chatbot": "https://github.com/DAGWorks-Inc/burr/tree/main/examples/multi-modal-chatbot",
"demo_conversational-rag": "https://github.com/DAGWorks-Inc/burr/tree/main/examples/conversational-rag",
"demo_counter": "https://github.com/apache/burr/tree/main/examples/hello-world-counter",
"demo_tracing": "https://github.com/apache/burr/tree/main/examples/tracing-and-spans/application.py",
"demo_chatbot": "https://github.com/apache/burr/tree/main/examples/multi-modal-chatbot",
"demo_conversational-rag": "https://github.com/apache/burr/tree/main/examples/conversational-rag",
}
return project_id_map.get(project_id, "")

Expand Down
2 changes: 1 addition & 1 deletion docs/examples/chatbots/gpt-like-chatbot.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
"<img src=\"https://github.com/user-attachments/assets/2ab9b499-7ca2-4ae9-af72-ccc775f30b4e\" width=\"100\" align=\"left\" /> + \n",
"<img src=\"https://cdn.mos.cms.futurecdn.net/VgGxJABA8DcfAMpPPwdv6a.jpg\" width=\"200\" align=\"center\"/>\n",
"\n",
"[https://github.com/apache/burr](https://github.com/apache/burr) by DAGWorks Inc. (YCW23 & StartX).\n",
"[https://github.com/apache/burr](https://github.com/apache/burr).\n",
"\n",
"Take🏠:\n",
"\n",
Expand Down
2 changes: 1 addition & 1 deletion examples/adaptive-crag/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,4 @@ We stop after `ask_assistant` to get the next `query` for `router`.
## Adaptive-CRAG 🧪
![statemachine](statemachine.png)

We have some of the core concepts from the [Burr Docs](https://burr.dagworks.io/concepts/) in a folder and we will add that to a `LanceDB` table for Hybrid Search. You can add as many tables as you want. Each table will be a different route for the `router`.
We have some of the core concepts from the [Burr Docs](https://burr.apache.org/concepts/) in a folder and we will add that to a `LanceDB` table for Hybrid Search. You can add as many tables as you want. Each table will be a different route for the `router`.
2 changes: 1 addition & 1 deletion examples/conversational-rag/graph_db_example/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ This will run the following two pipelines:
![ingest fighters](ingest_fighters.png)
![ingest fights](ingest_fights.png)

**Note:** [Hamilton](https://github.com/dagworks-inc/hamilton) also comes with a UI that you can use to visualize the pipeline and
**Note:** [Hamilton](https://github.com/apache/hamilton) also comes with a UI that you can use to visualize the pipeline and
track execution information about it. See hamilton_ingest.py or ingest_notebook.ipynb for more information.

## Ingest data using a notebook:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ def main():
# from hamilton_sdk import adapters
# tracker = adapters.HamiltonTracker(
# project_id=44, # modify this as needed
# username="elijah@dagworks.io",
# username="elijah@apache.org",
# dag_name="load_fighters",
# tags={"environment": "DEV", "team": "MY_TEAM", "version": "X"}
# )
Expand All @@ -60,7 +60,7 @@ def main():
# from hamilton_sdk import adapters
# tracker = adapters.HamiltonTracker(
# project_id=44, # modify this as needed
# username="elijah@dagworks.io",
# username="elijah@apache.org",
# dag_name="load_fights",
# tags={"environment": "DEV", "team": "MY_TEAM", "version": "X"}
# )
Expand Down
73 changes: 45 additions & 28 deletions examples/conversational-rag/graph_db_example/ingest_notebook.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,39 @@
"cells": [
{
"cell_type": "code",
"execution_count": null,
"id": "z27s4nl744c",
"source": "# Licensed to the Apache Software Foundation (ASF) under one\n# or more contributor license agreements. See the NOTICE file\n# distributed with this work for additional information\n# regarding copyright ownership. The ASF licenses this file\n# to you under the Apache License, Version 2.0 (the\n# \"License\"); you may not use this file except in compliance\n# with the License. You may obtain a copy of the License at\n#\n# http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing,\n# software distributed under the License is distributed on an\n# \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n# KIND, either express or implied. See the License for the\n# specific language governing permissions and limitations\n# under the License.",
"metadata": {},
"execution_count": null,
"outputs": []
"outputs": [],
"source": [
"# Licensed to the Apache Software Foundation (ASF) under one\n",
"# or more contributor license agreements. See the NOTICE file\n",
"# distributed with this work for additional information\n",
"# regarding copyright ownership. The ASF licenses this file\n",
"# to you under the Apache License, Version 2.0 (the\n",
"# \"License\"); you may not use this file except in compliance\n",
"# with the License. You may obtain a copy of the License at\n",
"#\n",
"# http://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing,\n",
"# software distributed under the License is distributed on an\n",
"# \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY\n",
"# KIND, either express or implied. See the License for the\n",
"# specific language governing permissions and limitations\n",
"# under the License."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1ad0bb7d-58db-44c5-aa17-7db4f9af9f59",
"metadata": {},
"outputs": [],
"source": [
"# install requirements\n",
"!pip install falkordb openai sf-hamilton[sdk,visualization] "
],
"outputs": []
]
},
{
"cell_type": "markdown",
Expand All @@ -26,7 +43,7 @@
"source": [
"# Ingestion Notebook\n",
"\n",
"In this notebook we see how to load data into FalkorDB using [Hamilton](https://github.com/dagWorks-Inc/hamilton)."
"In this notebook we see how to load data into FalkorDB using [Hamilton](https://github.com/apache/hamilton)."
]
},
{
Expand All @@ -39,11 +56,11 @@
"start_time": "2024-05-26T05:05:25.579866Z"
}
},
"outputs": [],
"source": [
"# load jupyter magic to help display Hamilton in a notebook\n",
"%load_ext hamilton.plugins.jupyter_magic"
],
"outputs": []
]
},
{
"cell_type": "markdown",
Expand All @@ -67,6 +84,7 @@
"start_time": "2024-05-26T05:12:04.295564Z"
}
},
"outputs": [],
"source": [
"%%cell_to_module -m load_fighters --display\n",
"\n",
Expand Down Expand Up @@ -138,8 +156,7 @@
" q = \"UNWIND $fighters as fighter CREATE (f:Fighter) SET f = fighter\"\n",
" graph.query(q, {'fighters': records})\n",
" return len(records)\n"
],
"outputs": []
]
},
{
"cell_type": "markdown",
Expand All @@ -165,6 +182,7 @@
"start_time": "2024-05-26T05:44:01.430713Z"
}
},
"outputs": [],
"source": [
"%%cell_to_module -m load_fights --display\n",
"\n",
Expand Down Expand Up @@ -252,8 +270,7 @@
"\n",
"def collect_writes(write_to_graph: Collect[str]) -> int:\n",
" return len(list(write_to_graph))"
],
"outputs": []
]
},
{
"cell_type": "markdown",
Expand All @@ -275,6 +292,7 @@
"start_time": "2024-05-26T05:44:05.448476Z"
}
},
"outputs": [],
"source": [
"import falkordb\n",
"from hamilton import driver\n",
Expand All @@ -286,36 +304,36 @@
"# Clear previous graph\n",
"if \"UFC\" in db.list_graphs():\n",
" g.delete()"
],
"outputs": []
]
},
{
"cell_type": "markdown",
"id": "f200ae21-1ab3-4bb8-ada1-876ddbd1fb5f",
"metadata": {},
"source": [
"### Note about the Hamilton UI\n",
"Hamilton comes with an [open source UI](https://hamilton.dagworks.io/en/latest/hamilton-ui/) that you can\n",
"Hamilton comes with an [open source UI](https://hamilton.apache.io/en/latest/hamilton-ui/) that you can\n",
"surface information about your Hamilton executions. If you pull the docker containers\n",
"locally and uncomment adding the HamiltonTracker, then you'll see runs logged to it."
]
"Hamilton comes with an [open source UI](https://hamilton.apache.org/en/latest/hamilton-ui/) that you can\n",
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"id": "ec7e41c83b69311c",
"metadata": {
"ExecuteTime": {
"end_time": "2024-05-26T05:44:33.832688Z",
"start_time": "2024-05-26T05:44:06.038046Z"
}
},
"outputs": [],
"source": [
"# if you have the Hamilton UI you can see progress:\n",
"# from hamilton_sdk import adapters\n",
"# tracker = adapters.HamiltonTracker(\n",
"# project_id=44, # modify this as needed\n",
"# username=\"elijah@dagworks.io\",\n",
"# username=\"elijah@apache.org\",\n",
"# dag_name=\"load_fighters\",\n",
"# tags={\"environment\": \"DEV\", \"team\": \"MY_TEAM\", \"version\": \"X\"}\n",
"# )\n",
Expand All @@ -332,25 +350,25 @@
")\n",
"\n",
"fighter_loader.execute([\"write_to_graph\"], inputs={\"graph\": g})"
],
"outputs": []
]
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"id": "3b703eeb5a360",
"metadata": {
"ExecuteTime": {
"end_time": "2024-05-26T05:49:42.579219Z",
"start_time": "2024-05-26T05:44:33.835479Z"
}
},
"outputs": [],
"source": [
"# if you have the Hamilton UI you can see progress:\n",
"# from hamilton_sdk import adapters\n",
"# tracker = adapters.HamiltonTracker(\n",
"# project_id=44, # modify this as needed\n",
"# username=\"elijah@dagworks.io\",\n",
"# username=\"elijah@apache.org\",\n",
"# dag_name=\"load_fights\",\n",
"# tags={\"environment\": \"DEV\", \"team\": \"MY_TEAM\", \"version\": \"X\"}\n",
"# )\n",
Expand All @@ -364,8 +382,7 @@
" .build()\n",
")\n",
"fights_loader.execute([\"collect_writes\"], inputs={\"graph\": g})"
],
"outputs": []
]
},
{
"cell_type": "code",
Expand All @@ -377,16 +394,16 @@
"start_time": "2024-05-26T05:49:42.580724Z"
}
},
"source": [],
"outputs": []
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"id": "7981e2c8294873e5",
"metadata": {},
"source": [],
"outputs": []
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down
4 changes: 2 additions & 2 deletions examples/hamilton-integration/notebook.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Use the 2-layer approach for a maintainable RAG system [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dagworks-inc/burr/blob/main/examples/hamilton-integration/notebook.ipynb) [![GitHub badge](https://img.shields.io/badge/github-view_source-2b3137?logo=github)](https://github.com/dagworks-inc/burr/blob/main/examples/hamilton-integration/notebook.ipynb)\n",
"# Use the 2-layer approach for a maintainable RAG system [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/dagworks-inc/burr/blob/main/examples/hamilton-integration/notebook.ipynb) [![GitHub badge](https://img.shields.io/badge/github-view_source-2b3137?logo=github)](https://github.com/apache/burr/blob/main/examples/hamilton-integration/notebook.ipynb)\n",
"\n",
"Ready-made solutions can get you started with GenAI, but building reliable product features with retrieval augmented generation (RAG) and LLM agents inevitably required custom code. This post shares the 2-layer approach to build a maintainable RAG application that will evolve with your needs. To illustrate these ideas, we will show how a typical RAG project might evolve.\n",
"\n",
Expand Down Expand Up @@ -676,7 +676,7 @@
"\n",
"In th next snippets, we refactor actions using Hamilton, a lightweight library to structure data transformations as directed acyclic graphs (DAGs). Hamilton uses the function and parameter names to infer the dependencies between functions and the graph structure.\n",
"\n",
"The next cell reimplements the `ingest_blog` action from `V2`. It uses `%%cell_to_module` from the Hamilton notebook extension to define a DAG in a single cell and view it ([see tutorial](https://github.com/DAGWorks-Inc/hamilton/blob/main/examples/jupyter_notebook_magic/example.ipynb))."
"The next cell reimplements the `ingest_blog` action from `V2`. It uses `%%cell_to_module` from the Hamilton notebook extension to define a DAG in a single cell and view it ([see tutorial](https://github.com/apache/hamilton/blob/main/examples/jupyter_notebook_magic/example.ipynb))."
]
},
{
Expand Down
6 changes: 3 additions & 3 deletions examples/image-telephone/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ that talks to itself to do something fun. The game is simple:
2. That caption is then provided to DALL-E, which generates an image based on the caption, which is saved to state.
3. The loop repeats -- and you have encoded the game of telephone!

Specifically, each action here in Burr is delegated to the [Hamilton](https://github.com/dagworks-inc/hamilton) micro-framework to run.
Specifically, each action here in Burr is delegated to the [Hamilton](https://github.com/apache/hamilton) micro-framework to run.
Hamilton is a great replacement for tools like LCEL, because it's built to provide a great SDLC experience, in addition
to being lightweight, extensible and more general
purpose (e.g. it's great for expressing things data processing, ML, and web-request logic). We're using
Expand Down Expand Up @@ -71,5 +71,5 @@ documentation for the Hamilton dataflows that are used: [captioning](https://hub
template by overriding values, or by copying the code and modifying it yourself in 2 minutes - see instructions on the [hub](https://hub.dagworks.io/).

## Hamilton code
For more details on the [Hamilton](https://github.com/dagworks-inc/hamilton) code and
this [streamlit app](https://image-telephone.streamlit.app) see [this example in the Hamilton repo.](https://github.com/DAGWorks-Inc/hamilton/tree/main/examples/LLM_Workflows/image_telephone)
For more details on the [Hamilton](https://github.com/apache/hamilton) code and
this [streamlit app](https://image-telephone.streamlit.app) see [this example in the Hamilton repo.](https://github.com/apache/hamilton/tree/main/examples/LLM_Workflows/image_telephone)
Loading
Loading