2023 Sky Camp

Friday, October 20, 2023 – Banatao Auditorium in Sutardja Dai Hall, UC Berkeley Campus

Sky Camp is where you can get exposure to specific Sky Computing research projects through talks and demos, as well as hands-on experience with systems and technologies for emerging applications.

Agenda

  • 9:00 AM – 10:00 AM: Breakfast
  • 10:00 AM – 10:15 AM: Opening Talk: Overview of Sky Camp
  • 10:15 AM – 11:45 AM: Session 1
    • SkyPilot
      • SkyPilot is a framework for running LLMs, AI, and batch jobs on any cloud, offering maximum cost savings, highest GPU availability, and managed execution. In the Sky Computing ecosystem, it is an intercloud broker system that sends jobs to the best cloud location and oversees their execution.
      • Slides, Tutorial, GitHub, Slack, Twitter, Video
    • FastChat
      • FastChat is an open platform for training, serving, and evaluating large language model based chatbots. It was used to train the Vicuna model and serve the Chatbot Arena demo. In this tutorial, we will demonstrate how to use FastChat’s distributed serving system to serve large language models with a WebUI and OpenAI-compatible API servers. We will also show you how to fine-tune Llama to build your own chatbots.
      • Tutorial, Video
  • 11:45 AM – 12:30 PM: Lunch
  • 12:30 PM – 2:15 PM: Session 2
    • Gorilla
      • Gorilla enables LLMs to use tools by invoking APIs. Given a natural language query, Gorilla comes up with the semantically- and syntactically- correct API to invoke. With Gorilla, we are the first to demonstrate how to use LLMs to invoke 1,600+ (and growing) API calls accurately while reducing hallucination.
      • Tutorial, Website, Slides, Video
    • DSPy
      • DSPy is a framework for solving complex tasks with language models and retrieval models. Instead of hard-coding lengthy prompts for specific models, you express a Pythonic program with declarative steps, and you let DSPy compile your program into automatic prompts (or finetunes) that are optimized for any LM, pipeline, metric, or data you want. This tutorial will include hands-on examples of using DSPy to solve complex NLP tasks.
      • GitHub, Slides, Video
    • FrugalGPT
      • FrugalGPT is a framework for using LLMs with better quality and lower cost simultaneously. By leveraging a plethora of LLMs strategically, FrugalGPT can match or surpass the performance of GPT-4 while incurring an order of magnitude lower cost. This tutorial will show concrete examples of using FrugalGPT to save the cost.
      • Tutorial, Slides, Video
  • 2:15 PM – 2:30 PM: Break
  • 2:30 PM – 3:30 PM: Session 3
    • Skyplane
      • Skyplane is a tool for blazingly fast bulk data transfers between object stores in the cloud. It provisions a fleet of VMs in the cloud to transfer data in parallel while using compression and bandwidth tiering to reduce cost. You can use Skyplane to transfer data between object stores within a cloud provider (e.g. AWS us-east-1 to AWS us-west-2), between object stores across multiple cloud providers (e.g. AWS us-east-1 to GCP us-central1), and between local storage and cloud object stores.
      • Slack, Slides, Video
    • SkyATC
      • SkyATC an end-to-end system that deploys LLM in any cloud that focuses on speed and elasticity. We believe one of bottlenecks for LLM in production is the speed of iteration and deployments. Currently it takes about 30 minutes to “boot” a replica of LLM on a public cloud VM instead. SkyATC tackles this problem by rethinking each the infrastructure layer and optimize them specifically for LLM workload.
      • Slides, Video
  • 3:30 PM – 3:45 PM: Break
  • 3:45 PM – 4:45 PM: Session 4
    • Accurately Detecting Antipatterns of Ray Programs with LLM and Static Analysis
      • Antipatterns in the programs built on Ray can lead to significant performance overhead. Our initial attempt to address this gap employed traditional static analysis and runtime debugging techniques but suffered from high rates of both false positives and false negatives due to program complexity. To overcome these limitations, we propose a new hybrid approach that incorporates LLM-based methods to augment traditional analysis techniques. Our experiments demonstrate that the proposed tool achieves near-zero rates of false positives and negatives while improving the performance.
      • Slides, Video

FAQ

Is there parking?

We strongly encourage public transportation or ride-sharing. However, we can provide parking passes for the nearby garage, Upper Hearst Parking Structure. The garages tend to fill up by around 10:00 AM, so please arrive early. If you will need a parking pass, please email us.

How do I register?

Registration is by invitation only. Invites will be sent beginning in mid-September.

Prerequisites / Technical Requirements

  • Laptop
  • “Modern” browser (Firefox, Chrome, Safari etc.)
  • Experience programming in Python in notebook environment
  • Basic understanding of AI/ML concepts (e.g., training, validation, linear models)

Lodging

Some local hotel options are listed on our contact page.

Contact

For questions, email skycamp [at] berkeley.edu