Deploy Haystack

Haystack is an open source framework for building applications that use large language models (LLMs). Instead of requiring you to write custom orchestration logic for every use case, it provides a structured way to assemble AI functionality through reusable components connected in pipelines. This approach makes it well-suited for creating retrieval-augmented generation (RAG) systems, AI agents, search experiences, chat applications, and other LLM-driven solutions.

Deploying a Quick Deploy App

Akamai Quick Deploy Apps let you easily deploy software on a Compute Instance using Cloud Manager. See Get Started with Quick Deploy Apps for complete steps.

1. Log in to Cloud Manager and select the Quick Deploy Apps link from the left navigation menu. This displays the Linode Create page with the Marketplace tab pre-selected.

2. Under the Select App section, select the app you would like to deploy.

3. Complete the form by following the steps and advice within the Creating a Compute Instance guide. Depending on the Quick Deploy App you selected, there may be additional configuration options available. See the Configuration Options section below for compatible distributions, recommended plans, and any additional configuration options available for this Quick Deploy App.

4. Click the Create Linode button. Once the Compute Instance has been provisioned and has fully powered on, wait for the software installation to complete. If the instance is powered off or restarted before this time, the software installation will likely fail.

To verify that the app has been fully installed, see Get Started with Akamai Quick Deploy Apps > Verify Installation. Once installed, follow the instructions within the Getting Started After Deployment section to access the application and start using it.

Configuration Options

• Supported distributions: Ubuntu 24.04 LTS
• Recommended plan: All plan types and sizes can be used.

Haystack Options

Limited Sudo User

You need to fill out the following fields to automatically create a limited sudo user, with a strong generated password for your new Compute Instance. This account will be assigned to the sudo group, which provides elevated permissions when running commands with the sudo prefix.

• Limited sudo user: Enter your preferred username for the limited user. No Capital Letters, Spaces, or Special Characters.

  Locating The Generated Sudo Password

  A password is generated for the limited user and stored in a .credentials file in their home directory, along with application specific passwords. This can be viewed by running: cat /home/$USERNAME/.credentials

  For best results, add an account SSH key for the Cloud Manager user that is deploying the instance, and select that user as an authorized_user in the API or by selecting that option in Cloud Manager. Their SSH pubkey will be assigned to both root and the limited user.

• Disable root access over SSH: To block the root user from logging in over SSH, select Yes. You can still switch to the root user once logged in, and you can also log in as root through Lish.

  Accessing The Instance Without SSH

  If you disable root access for your deployment and do not provide a valid Account SSH Key assigned to the authorized_user, you will need to login as the root user via the Lish console and run cat /home/$USERNAME/.credentials to view the generated password for the limited user.

Getting Started after Deployment

Testing Python SDK

Once the deployment is complete, the haystack-ai library should already be installed on your instance. This allows you to import the library into your software. To get started:

1. Create an example directory called science.

   mkdir science

2. Create a test Python file called agent.py that allows you to use our AI model.

   cd science
   vim agent.py
   

3. Enter the following content into the agent.py Python file.

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

   from haystack.components.agents import Agent from haystack.components.generators.chat import OpenAIChatGenerator from haystack.dataclasses import ChatMessage from haystack.tools import ComponentTool from haystack.utils import Secret agent = Agent( chat_generator=OpenAIChatGenerator( api_base_url="http://localhost:8000/v1", api_key=Secret.from_token("EMPTY"), model="Qwen/Qwen3-14B-AWQ", ), system_prompt="You are a helpful assistant that can search the web for information.", ) result = agent.run( messages=[ChatMessage.from_user("What is Haystack AI?")] ) print(result["last_message"].text)

4. Once you save the file, execute it with the following command.

   python3 agent.py

This example uses a self-hosted model exposed via the LLM’s API. If you want to use a provider model, refer to the Haystack documentation.

More Information

You may wish to consult the following resources for additional information on this topic. While these are provided in the hope that they will be useful, please note that we cannot vouch for the accuracy or timeliness of externally hosted materials.

• Haystack Documentation