# Welcome to Q Blocks [**Q Blocks**](https://www.qblocks.cloud/) is a new kind of GPU cloud solution designed for AI model training, tuning, and deployment workloads. Our platform is built using a distributed computing approach, allowing us to offer upto 80% more affordable and scalable GPU cloud services compared to other cloud providers. With Q Blocks, you can easily port your existing workloads to our platform without any code changes. All GPU instances are containerised, ensuring seamless integration with your current workflows. We have state of the art Nvidia GPU instances to offer in single and multi-GPU configurations to help you train and deploy small as well as large ML models. Our platform is also highly decentralised, which means we can orchestrate container-native workloads at scale across a grid of heterogeneous machines, including consumer and data center nodes. At Q Blocks, we are committed to providing application-specific pipelines that enable the execution of the entire value chain for developers. Today, we offer a pipeline specifically designed for training and fine-tuning machine learning models. We understand the importance of deploying and managing GPU instances at scale, and that's why we also offer Rest APIs to streamline the process of deploying and managing GPU instances for workloads like AI model inference. Overall, Q Blocks is a powerful, affordable, and scalable solution for companies looking to streamline their AI model training, tuning, and deployment workflows. # GPU Computing at Scale Q Blocks has been designed primarily to enable access of scalable and very affordable GPU computing power for AI/ML workloads. We are not a cloud platform that offers 100s of services like VPC or storage services. We focus today on computing services; more specialised for GPU compute. Unlike a traditional cloud platform with centralised / large data centers and prohibitively expensive access of infrastructure; Q Blocks developed an alternate approach of accessing GPU power by partnering with thousands of GPU server owners across the world. We call these partners as hosts and these hosts are of 2 types: 1. Small facilities with 20-100 GPU servers 2. Tier 2 - 3 Data centers with 1000+ GPU servers. This approach helps us offer 3 fundamental value additions for AI/ML businesses: 1. Scalable access of GPU servers with high availability 2. High optionality of GPU types to choose from On our platform we are able to offer a ton of GPU options to choose from: A screenshot of Q Blocks GPU instance launch dashboard A large variety of GPUs from 80GB VRAM to 8GB VRAM are available. If we do a straightforward price to price comparison for some of the most in-use GPU servers like Tesla V100 16GB GPUs between AWS and Q Blocks then we will notice a straightaway 50% cost reduction: | Parameter | AWS | Q Blocks | | ------------- | ----------------------- | ----------------------- | | Instance Type | P3.2xlarge - Tesla V100 | QB16-v1-n1 - Tesla V100 | | Cost | $3.06/hr | $1.5/hr | 1. We don't charge for egress bandwidth. 2. We offer Technical support at no additional cost. With the Data Center nodes we support Tier 2 grade uptime and reliability of GPU instances. # Launch a Q Blocks GPU instance Q Blocks offers support for launching and managing GPU instances via 2 methods: 1. [Using Dashboard:](/launch-a-q-blocks-gpu-instance/using-dashboard-ui) 1. Dashboard offers an easy to use graphical interface to launch and manage GPU instances. 2. This method is easy and is great for launching one or two GPU instances 2. [Using Rest APIs:](/launch-a-q-blocks-gpu-instance/using-rest-apis) 1. Our Rest APIs offer scalable access to launch and manage GPU instances programmatically. 2. This method requires programming skillset. 3. These APIs can be integrated into your application or business logic for on-demand access of GPUs based on your usage needs. Depending on your use-case requirements you may choose either of the above mentioned methods. Both offer different capabilities specific to scaling needs of your business. # Using Dashboard UI To get started follow the steps below: 1. #### Go to [Create Instance](https://www.qblocks.cloud/client/v2/create-instance) Page: 2. #### Select a GPU Type: 1. Different GPUs carry different number of processing cores and memory. 2. Based on your ML model’s size and number of parameters, you can choose a specific GPU type 3. **Select required data storage:** 1. Q Blocks instances offer fast IO in GPU instances with NVMe/SSD storage access for best performance 4. **Choose an AI Framework Image:** 1. Images help you pre-configure your instance with AI frameworks such as TensorFlow, PyTorch or Keras 2. They also come with out-of-the-box support for CUDA and GPU drivers. 3. If you don't a specific framework driven environment then choose base Ubuntu Image. 4. All these environments offer superuser (sudo su) access with support for installation and management of python pip packages, conda environments etc. You may consider it as a linux based GPU instance which can be modified as per your software needs. Once the configuration is selected, click on “Create Instance”. > ✅ The GPU powered computing instance should be accessible in 1 - 5 minutes. ### Checkout this video for quick walkthrough: {% embed url="" %} We hope to see you make something wonderful with Q Blocks GPU instances. #### Technical support: For any query or issues in getting started, reach us out at for quick resolution. # Using Rest APIs Q Blocks offers a detailed swagger documentation for launching and managing GPU instances via our Rest APIs. To get started, you'd need 2 information pieces: 1. [Your Q Blocks Client ID](#q-blocks-client-id) 2. [An Authentication Token](#authentication-token) #### Q Blocks Client ID: Your Q Blocks client ID is a unique string that helps us identify your account when you send API requests. Access your Q Blocks `client_id` [here](https://www.qblocks.cloud/client/v2/account/auth). #### Authentication Token: A bearer auth token is required in header of each API request sent by you as that'd help us authenticate your requests. Generate an authentication token [here](https://www.qblocks.cloud/client/v2/account/auth). 1. Click on "Create new Auth Token" button. 2. Select the permissions you'd like to add to your auth token. 3. Click "Generate Auth Token" button. Store the newly generated auth token somewhere safe. After you have successfully generated an auth token and copied your client\_id, you are all set to send requests to our Rest APIs for launching and managing GPU instances. Please refer to below links for performing different actions: 1. [View available GPU types and Launch a GPU instance](https://controlapi.qblocks.cloud/docs?urls.primaryName=Instance%20API%20\(Creation\)) 2. [Perform instance actions like: Start, Stop, Restart, Command run, ](https://controlapi.qblocks.cloud/docs?urls.primaryName=Instance%20Action%20API#/Action)[Delete](https://controlapi.qblocks.cloud/docs?urls.primaryName=Instance%20Action%20API#/Action) 3. [Fetch instance information like: Status, Logs, Access credentials](https://controlapi.qblocks.cloud/docs?urls.primaryName=Instance%20Action%20API#/Info) With these APIs, you are all set to access scalable GPU infrastructure on Q Blocks at upto 60% low cost for your AI/ML workloads. #### Technical support: For any query or issues in getting started, reach us out at for quick resolution. # GPU Instance Pricing Q Blocks is one of the most cost effective GPU computing platforms designed to enable higher price to performance ratio for AI/ML dev teams. Our optimised GPU instances offer very cost effective pricing for AI/ML workloads. We have 2 types of GPU instances: | Parameter | Consumer GPU nodes | Data Center GPU nodes | | --------------------------------- | ------------------ | -------------------------- | | **Pricing (compared to AWS/GCP)** | Upto 80% low cost | Upto 50% low cost | | **Security** | Basic security | Premium certified security | | **Uptime SLA** | Upto 99% | Tier 2/3 DC uptime SLA | | **Pause/Resume support** | Not supported | Supported | We offer discounted pricing on monthly rentals as well. ### On-demand Pricing for Data Center GPU nodes:
Block NameGPU TypeGPU MemoryPricing/hour (in USD)
QB8-v1-n11x Quadro RTX 40008GB0.47
QB16-v2-n11x Tesla V10016GB1.3
QB16-v1-n11x RTX A400016GB0.95
QB16-v1-n22x RTX A40002x 16GB1.8
QB16-v1-n44x RTX A40004x 16GB3.6
QB24-v2-n11x RTX A500024GB1.25
QB24-v2-n22x RTX A50002x 24GB2.5
QB24-v2-n44x RTX A50004x 24GB5
QB40-v1-n11x A100 PCIE1x 40GB2.75
QB48-v1-n11x RTX A600048GB2.02
QB48-v1-n22x RTX A60002x 48GB4.1
QB80-v1-n11x A100 SXM80GB3
QB80-v1-n22x A100 SXM2x 80GB5.5
QB80-v1-n44x A100 SXM4x 80GB10.5
QB80-v2-n11x A100 PCIE1x 80GB3.3
QB80-v2-n22x A100 PCIE2x 80GB6.25
{% hint style="info" %} Pricing may change over time and is subject to availability of GPU types. {% endhint %} ### Pricing for consumer nodes: For consumer nodes, kindly check the pricing directly in your dashboard. For any queries please don't hesitate to reach us on # Fine-tuning Falcon 7B/40B LLM Falcon is a family of open-source large language models (LLMs) with 7 billion and 40 billion parameters trained on one trillion tokens. We can fine-tune Falcon on Q Blocks cloud by running these commands for installation and execution: GPU configuration: * We would recommend choosing a 40GB or higher GPU such as 1x A100 40GB/80GB, 1x A6000 or 2x A100 80GB from the Data center nodes option on [Q Blocks platform](https://www.qblocks.cloud/client/v2/create-instance) while launching a GPU instance. **Install miniconda** ```bash # Download latest miniconda. wget -nc https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh # Install. -b is used to skip prompt bash Miniconda3-latest-Linux-x86_64.sh -b # Activate. eval "$(/home/qblocks/miniconda3/bin/conda shell.bash hook)" # (optional) Add activation cmd to bashrc so you don't have to run the above every time. printf '\neval "$(/home/qblocks/miniconda3/bin/conda shell.bash hook)"' >> ~/.bashrc ``` **Setup env** Install using the yaml file: ```bash # Create and activate env. -y skips confirmation prompt. conda create -n falcon-env python=3.9 -y conda activate falcon-env # newest torch with cuda 11.8 conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia # Install other dependencies pip install -U accelerate einops sentencepiece git+https://github.com/huggingface/transformers.git && \ pip install -U trl git+https://github.com/huggingface/peft.git && \ pip install scipy datasets bitsandbytes wandb ``` **Start the run** Download script and execute it in conda environment: 
# Download finetuning script
wget https://qbcontent.nyc3.cdn.digitaloceanspaces.com/finetuning/finetune-falcon.py

eval "$(/home/qblocks/miniconda3/bin/conda shell.bash hook)"
conda activate falcon-env

# Single GPU, falcon 7B, 4bit quantization
torchrun --nnodes 1 --nproc_per_node 1 \
ft.py \
-m ybelkada/falcon-7b-sharded-bf16 \
-q 4bit

# 8x GPUs, falcon 40B, 8bit quantization
torchrun --nnodes 1 --nproc_per_node 2 \
finetune-falcon.py \
-m tiiuae/falcon-40b \
-q 4bit
More parameters can be specified such as: `--dataset_name` `--steps` `--batch_size_per_device` # IAM: Share access with team Q Blocks now supports account sharing with your team members so they can sign-in through their account and perform actions on your resources based on the permissions granted to them by you. Currently, the following permissions can be granted to team members: 1. **Instance access:** Allows them to view and access your instances. 2. **Instance stop:** Allows them to stop a running instance 3. **Instance start:** Allows them to start a paused instance 4. **Instance delete:** Allows them to delete an instance (CRITICAL) Based on your specific requirements and organisation design, you may grant access privileges to different team members in your main account. > **Pre-requisite:** > > **Before you add a team member to your account, they must have a registered / active account on Q Blocks.** To add a team member, perform the following steps: 1. Go to "IAM Access" section on Settings page. 2. Click on "Add a new team member" button. 3. Enter your team member's email ID that they used for registering their account on Q Blocks and the privileges you'd like to grant them. 4. Click on "Submit" button. This should add a teammate to your account with your granted privileges. Here's a short video to walk you through the process: {% embed url="" %} Once you've successfully added a teammate, you may remove them whenever you feel is necessary by clicking on "Remove" button alongside their email ID. Upon adding a teammate, they should be able to access instances or perform actions on them through their dashboard. #### Technical support: For any query or issues in getting started, reach us out at for quick resolution. # Q Blocks How To Guide Q Blocks is a decentralized computing infrastructure platform designed to serve ML workloads at scale. To enable full feature utilization, we provide you a quick How-To Guide. The guide contains documentation to help you quickly get started with functionalities that Q Blocks instances offer out of the box: *** ### Basic Instance functionality: *** [ Create a new user](/q-blocks-how-to-guide/create-a-new-user) [ Upload data using SCP command](/q-blocks-how-to-guide/upload-data-using-scp-command) [ Use Visual Studio Code with Q Blocks instances](/q-blocks-how-to-guide/use-visual-studio-code-with-q-blocks-instances) [ Port forwarding to run web services](/q-blocks-how-to-guide/port-forwarding-to-run-web-services) [ Launch Jupyter Hub in Q Blocks Instance](/q-blocks-how-to-guide/launch-jupyter-hub-in-q-blocks-instance) [ Upgrade Instance CUDA to version 12.2](/q-blocks-how-to-guide/upgrade-cuda-to-v12.2) *** ### Machine learning tooling setup: *** [ Launch TensorBoard in Q Blocks instance](/q-blocks-how-to-guide/launch-tensorboard-in-q-blocks-instance) [ Setup Horovod and OpenMPI in Q Blocks Instance](/q-blocks-how-to-guide/setup-horovod-and-openmpi-in-q-blocks-instance) [ Setup AIM for ML experiment tracking](/q-blocks-how-to-guide/setup-aim-for-ml-experiment-tracking) [ Disco Diffusion AI Art on Q Blocks](/q-blocks-how-to-guide/disco-diffusion-ai-art-on-q-blocks) [ Stable Diffusion Text to Image GPU server on Q Blocks](/q-blocks-how-to-guide/stable-diffusion-text-to-image-gpu-server-on-q-blocks) ## Supported by Pro Instances: *** ### Docker setup: *** [ Setup Docker with Nvidia GPU support](/q-blocks-how-to-guide/setup-docker-with-nvidia-gpu-support) [ Enable port forwarding on a Docker container in Q Blocks instance](/q-blocks-how-to-guide/enable-port-forwarding-on-a-docker-container-in-q-blocks-instance) [Run production ready lightweight kubernetes using K3s in Q Blocks instance](/q-blocks-how-to-guide/run-production-ready-lightweight-kubernetes-using-k3s-in-q-blocks-instance) # Create a new user Ubuntu / Linux is the base operating system in Q Blocks computing instances. Linux is a multi-user system: This means that more than one person can interact with the same system at the same time. As a system administrator, you have the responsibility to manage the system’s users and groups by creating and removing users and assign them to different groups. #### Create new user in Q Blocks instance: We have created a one step easy process for you to create new users in your instance. To create new user and setup its profile with right permissions, execute this command: ```bash wget -qO - 'https://www.qblocks.cloud/resources/add-new-user.sh' --no-check-certificate | bash -s USERNAME PASSWORD ``` * *Make sure to replace USERNAME and PASSWORD in the above command with your desired values* The above command would do the following tasks: 1. Create a new user for you with your desired username and password 2. Grant the SSH access to the new user **Note:** If you don't want to use the above mentioned script then you can manually create a new user by running the standard bash commands for adding user in Linux. In this case, to grant SSH access to your new user you will have to add the AllowUser setting in SSH config. The SSH access would otherwise be restricted as a security measure to stop any unauthenticated access. If you still face issues setting it up then don't hesitate to reach out to us at: [support@qblocks.cloud](mailto:support@qblocks.cloud?subject=Unable%20to%20launch%20TensorBoard) # Upload data using SCP command SCP (secure copy) is a command-line utility that allows you to securely copy files and directories between two locations. With scp , you can copy a file or directory: From your local system to a remote system. It uses the same authentication and security as it is used in the Secure Shell (SSH) protocol. SCP is known for its simplicity, security and pre-installed availability. When you launch an instance on Q Blocks, you can access it either through Jupyter Lab or through Terminal SSH. SSH access to your Q Blocks instances is controlled via: 1. Password, and 2. Secure public private key pair While accessing your instance, you can download your private SSH key or use password for authentication. For the sake of this SCP example, we will use SSH Key based authentication. To get started, please follow the steps mentioned below: 1. Go to the instances page in your Q Blocks Dashboard. 2. Click on "Select an action" button on your instance and then choose the "Connect" option 3. Copy the SSH port of your instance as shown in the image below:
4. Copy the Hostname of your instance: * In the above image, you can see our example hostname is: jeu.qblocks.cloud. * It will be different in your case but you need to copy your hostname for running scp command. 5. Go to your terminal and execute the following scp command: ```bash scp -i /PATH/TO/SSH/KEY -P SSH_PORT /PATH/TO/LOCAL_FILE qblocks@HOSTNAME:/home/qblocks/ ``` 6. In the above command we specify the following: * `/PATH/TO/SSH/KEY` for using it as authentication. * `SSH_PORT` is our instance's SSH Port. * `/PATH/TO/LOCAL_FILE` is the source path of the file that will be uploaded using SCP. You may even specify a folder path for upload. * `qblocks` is the default user of your instance. You can create new users as well in your instance and use its password or ssh key for scp. * `HOSTNAME` is your instance's hostname. * `/home/qblocks/` is the destination path where you want the data to be uploaded in your instance. You may specify any particular existing path in your instance as a destination path. This should help you quickly upload any file or data to your Q Blocks instance using scp. If you still face issues then don't hesitate to reach out to us at: [support@qblocks.cloud](mailto:support@qblocks.cloud?subject=Unable%20to%20launch%20TensorBoard). # Use Visual Studio Code with Q Blocks instances {% hint style="info" %} Setup time less than 5 minutes {% endhint %} ***Visual Studio Code*** is a code editor redefined and optimized for building and debugging modern web and cloud applications. It is commonly referred to as VS Code and is a source-code editor made by Microsoft for Windows, Linux and macOS. Q Blocks instances can be easily accessed through VSCode to support coding and development setup for your teams. #### Pre-requisites: * Enable TCP Forwarding in Q Blocks instance sshd\_config * Set Host Information in local system’s SSH config ### 1. Enable TCP Forwarding in Q Blocks instance sshd\_config: Visual Studio Code needs TCP Forwarding to work properly. TCP forwarding is by default disabled in Q Blocks instance sshd configuration file for security reasons. To change this setting, run the following commands as qblocks user in your Q Blocks instance: ```bash sudo sed -i 's/AllowTcpForwarding no/AllowTcpForwarding yes/g' /etc/ssh/sshd_config sudo systemctl restart sshd ``` This will Allow TCP forwarding and then restart the SSH Daemon. ### **2. Set Host Information in local system’s SSH config:** SSH config file contains the information of the remote hosts that you want to access from your local computer. VScode saves these remote hosts in the “REMORE EXPLORER” tab, so you don’t have to re-enter the ssh information next time and thus access the remote host from the VSCode on your local system easily. Simply add the following the information in your ssh config file present at: \~/.ssh/config location ```bash Host qblocks-instance HostName HOST_URL IdentityFile /PATH/TO/KEYFILE User USER Port SSH_PORT ``` Information such as SSH\_PORT, HOST\_URL, USER can be easily obtained by clicking on “Connect” option on your instance and then selecting the SSH information. As seen in the example image below: * HOST\_URL = jeu.qblocks.cloud * SSH\_PORT = 57231 * USER = qblocks (or any new user that you created) Please note: This is an example. The information will be different for your instance.
{% hint style="success" %} Please ensure the ssh key has 600 permissions set. {% endhint %} 🚀 Once configured, you can simply go to your Visual studio application and then select the qblocks-instance ssh target under the “REMORE EXPLORER” tab and try connecting to it. Please reach out to us at if you face any difficulty. # Port forwarding to run web services {% hint style="info" %} Read time: 4 minutes {% endhint %} Q Blocks instances offer support for running web services with public access. This is enabled through port forwarding. A web service can be forwarded at an open port of the Q Blocks instance and then accessed publicly. > 🔐 Due to security reasons, all ports are not open. Only some random ports can be opened. The pre-configured open ports of your instance can be viewed as follows: 1. Go to your [Active Instances](https://www.qblocks.cloud/client/v2/instances) Page. 2. Click on the dropdown icon on your instance row to view instance info. 3. **Extra Ports** section displays the open port as seen below:
> 🗒️ Copy the **Extra port** and **Host** values ### Example of a web service: As an example, let us run **Tensorboard** web service in our instance. Tensorboard is a Machine learning experiment tracking software that can be launched as a web service and then accessed in your browser. Run the following commands in your instance’s terminal: ```bash pip install tensorboard tensorboard --logdir LOG_FOLDER --host=0.0.0.0 --port EXTRA_PORT ``` > Replace EXTRA\_PORT with the Port number copied above. This should launch the service and it should be accessible at the following URL: ```html http://HOST:EXTRA_PORT ``` The same way, you can launch any web service in your instance at a provided port. **Please see:** > *Any application running at a forwarded port is accessible to the public. Right precautions with auth token or password access must be implemented to secure the applications.* If you still face issues setting it up then don't hesitate to reach out to us at: [support@qblocks.cloud](mailto:support@qblocks.cloud?subject=Unable%20to%20launch%20TensorBoard) # Launch Jupyter Hub in Q Blocks Instance {% hint style="info" %} Setup Time: 5 minutes {% endhint %} Q Blocks instances offer Jupyter Lab access out of the box. It is great for quickly spawning up multiple notebooks, python console and terminal access within your browser. It also offers support for extensions that add superpowers to your notebooks. However, there is another offering from the Jupyter community called as Jupyter Hub. Unlike Notebooks or Lab, Jupyter Hub lets you have multiple user access. If you are looking for extending access within your team to multiple users so they can have their isolated environments for running and managing notebooks then Hub is the best solution for you. So, let’s get started! ## Follow the 2 step process: #### 1. Install Jupyter Hub: * *Run this command as qblocks user:* ```bash wget 'https://www.qblocks.cloud/resources/setup-jupyterhub.sh' --no-check-certificate -O - | bash ``` > This script will install required dependencies and Jupyter hub software. #### 2. Launch Jupyter Hub: * *See this* [*guide*](https://www.notion.so/Port-forwarding-to-run-web-services-eefe92a817fa4deeb462c1b72b985164?pvs=21) *on where to find **Extra Port** and **Host information.*** * *Run this command as qblocks user:* ```bash sudo jupyterhub --ip 0.0.0.0 --port EXTRA_PORT ``` > EXTRA\_PORT is the extra port shown under More info section of your instance. #### Jupyter Hub Access: Jupyter hub can be accessed at this URL: ```bash http://HOST:EXTRA_PORT ``` > Replace Host and Extra port with the values shown in More info section of your instance. See above mentioned guide to find Extra port and Host information. Jupyter Hub URL can be shared across your team and each user account in your instance can access Jupyter Hub with their respective credentials. See this section on [How to create a new user account in Q Blocks instance](/q-blocks-how-to-guide/create-a-new-user). ## Advanced Settings (Optional): You may read further if you want to add an SSL certificate to access Jupyter Hub over HTTPS. Jupyter Hub accepts a python configuration file in which you can specify all the necessary conditions like port mapping and security certificates. ### **Steps to follow:** 1. **Generate or use an existing SSL certificate.** * Store the certificate files in `/home/qblocks/` as ssl.key and ssl.crt. * If you don’t have an SSL certificate, it can be generated using OpenSSL ([example](https://stackoverflow.com/a/14267011)). > Please Note: > > A self signed OpenSSL certificate may not be secure enough and we would recommend you to follow the industry standard practices to access your web services securely. 2. **Create a Jupyter Hub configuration file** * Store this config file as `/home/qblocks/jupyterhub_config.py`: > *Make sure to replace* EXTRA\_PORT *with your instance’s extra port* ```python # jupyterhub_config.py file c = get_config() import os pjoin = os.path.join # Allows multiple single-server per user c.JupyterHub.allow_named_servers = True c.JupyterHub.bind_url = 'http://0.0.0.0:EXTRA_PORT' # https certificate setup c.JupyterHub.ssl_key = '/home/qblocks/ssl.key' c.JupyterHub.ssl_cert = '/home/qblocks/ssl.crt' ``` > More info on Jupyter Hub config available [here](https://jupyterhub.readthedocs.io/en/stable/howto/configuration/config-ghoauth.html). 3. **Launch Jupyter Hub with configuration file:** ```bash sudo jupyterhub --config=/home/qblocks/jupyterhub_config.py ``` > Visit [https://HOST:EXTRA\\\_PORT](https://docs.qblocks.cloud/q-blocks-how-to-guide/https:/HOST:EXTRA\\_PORT) to access Jupyter Hub over https If you still face issues setting it up then don't hesitate to reach out to us at: [support@qblocks.cloud](mailto:support@qblocks.cloud?subject=Unable%20to%20launch%20TensorBoard) # Launch TensorBoard in Q Blocks instance Tensorboard comes pre-installed in the Q Blocks instances that are launched with Tensorflow based AI Images. But if you have launched a Q Blocks instance with a different AI image other than Tensorflow based then you can simply install Tensorboard with the following command: ```bash pip install tensorboard ``` Once you have launched the instance and Tensorboard is available in it, you can follow the next steps to get started with using it: 1. Click on the "More info" button for your instance on the Instances page in your Dashboard. 2. Copy the **Extra Port** and **Host URL** as shown in the image:
3. Now, access your instance via Terminal or Jupyter Lab and run the following command in your instance to launch Tensorboard: 
tensorboard --logdir LOG_FOLDER --host=0.0.0.0 --port EXTRA_PORT
> Replace EXTRA\_PORT with the Port number copied in last step 3. Now head over to your browser and access your TensorBoard running at the following URL: ```bash http://HOST_URL:EXTRA_PORT ``` #### For example: {% hint style="info" %} If we ran TensorBoard application in our instance with information provided in the above image then we would see it running at the following URL: {% endhint %} This should help you get started with running applications like TensorBoard that needs access via browser in your instance. If you still face issues setting it up then don't hesitate to reach out to us at: [support@qblocks.cloud](mailto:support@qblocks.cloud?subject=Unable%20to%20launch%20TensorBoard) # Setup Horovod and OpenMPI in Q Blocks Instance Horovod is a free and open-source software framework for distributed deep learning training using TensorFlow, Keras, PyTorch, and Apache MXNet. Horovod is hosted under the Linux Foundation AI. Setting up Horovod in your Q Blocks instance is as simple as running a single bash command. We have prepared a quick and easy to use bash script that installs and configures Horovod along with openMPI in your instance. Horovod uses openMPI for message passing during the distributed training process. {% hint style="info" %} Installation and setup will take anywhere between 5-10 minutes. {% endhint %} #### Follow the steps below for successful installation: 1. Go to your Q Blocks instance and open the bash terminal in it. 2. Run the following command: ```bash wget https://www.qblocks.cloud/resources/setup-horovod.sh --no-check-certificate -O - | sh ``` #### The above command would perform the following tasks: * Install any missing dependencies to build OpenMPI * Install OpenMPI * Install Horovod with support NCCL, Tensorflow and PyTorch #### Installation can be verified by running the command: ```bash horovodrun --help ``` If you face issues with the installation then reach out to us at . # Setup AIM for ML experiment tracking {% hint style="info" %} Setup time: 2 minutes {% endhint %} ## What is AIM? Aim is an open-source, self-hosted ML experiment tracking tool. It offers a beautiful and performant UI to track 1000s of training runs and compare them under a single dashboard. AIM UI also offers an SDK to query your runs' metadata programmatically. {% embed url="" %} > More info available at AIM’s [GitHub Repo](https://github.com/aimhubio/aim) **AIM** also outperforms **Tensorboard** in the following ways: 1. TensorBoard becomes really slow and hard to use when a few hundred training runs are queried / compared whereas AIM is built to handle 1000s of training runs. 2. TensorBoard doesn't have features to group, aggregate the metrics whereas with AIM you can search, group, aggregate via params - deeply explore all the tracked data (metrics, params, images) on the UI. ### Why use AIM in your Q Blocks GPU instances? Q Blocks GPU powered instances are designed to make your work as an ML engineer much easier by offering pre-configured instances with different ML environments and thus help you quickly get started with your ML model training runs and swiftly perform 1000s of experiments. With AIM, you can now track 1000s of experiments under one roof. Thus, making your job even more easier and efficient. ### Pre-requisites for AIM Setup: 1. **Launch a GPU instance on Q Blocks:** 1. Launching a GPU powered instance on Q Blocks in fairly straightforward. 2. If not sure how to proceed, use this guide: [Launch a GPU instance on Q Blocks](https://www.notion.so/525c274558064bc7890a6f5704b193c0?pvs=21) 2. **Get your instance’s open-port:** 1. Open port represents a port on which you can run any service and then access it publicly. 2. Once your instance is launched, you will see “More Info” dropdown on your instance bar. 3. Click on “More Info” Dropdown to copy the **Extra Port** and **Host URL** information. ### Setup AIM Dashboard: AIM can be installed and launched within just 2 lines of code. Run the following commands in your instance’s bash terminal. #### Install AIM: ```bash pip install aim ``` #### Launch AIM Service: ```bash aim up -h 0.0.0.0 -p Extra_Port ``` > Replace Extra\_Port with the Port you have copied from your instance's "More info" section. ### Access AIM Dashboard: Once the AIM service has been launched, its dashboard will become available at the following URL: ```bash http://Host_URL:Extra_Port ``` It should looking something like this:
Once the dashboard is up and running, you can simply call AIM in your python code and then easily log the experiments that can be viewed and compared in the dashboard. ## AIM Integrations: AIM can easily be imported in your python code. ```python import aim ``` AIM also offers direct integration with frameworks such as: 1. Pytorch Lightening 2. HuggingFace 3. Keras 4. XGBoost Refer to [this section](https://github.com/aimhubio/aim#integrations) for more information on how to easily import AIM into your codebase. If you face issues with the installation then reach out to us at . # Disco Diffusion AI Art on Q Blocks Read time: 2 minutes ## What is Disco Diffusion? Disco Diffusion is **a free tool that you can use to create “AI” generated art**. You can create machine learning generated images and videos with it.

Text Prompt: An oil painting of an afternoon in a green field with a red house on it, by Thomas Kinkade

> 🔥 So let’s get started with Disco Diffusion using DiscoArt on Q Blocks! ### Steps to run Disco Diffusion on Q Blocks: 1. Launch a [Q Blocks GPU Instance](https://www.qblocks.cloud/client/v2/create-instance) with “Disco Diffusion” AI Image 2. Access the instance through Jupyter Lab and open the Disco Diffusion Notebook. 3. Change the input text to a desired prompt and run Disco diffusion cell to generate artistic AI renderings for you. That’s it! 😀👨‍🎨 **A quick visual tutorial:** {% embed url="" %} ### Extra info: Disco Diffusion requires powerful GPUs to get faster results and the environment setup can be complicated and difficult to get started with. To make the process a lot easier, Q Blocks integrated Jina AI’s recent open source application Discoart - An easy to use disco diffusion software. So Q Blocks GPU integrated experience with DiscoArt resulted in: ✅ Very easy one click setup ✅ Quick Disco Diffusion AI art generation within 2 minutes. ✅ More affordable GPU driven rendering using Q Blocks > 👨‍🎨 **Examples generated using DiscoArt on Q Blocks:**
# Stable Diffusion Text to Image GPU server on Q Blocks
Stable Diffusion is a state of the art text-to-image model that generates high fidelity images from your text prompts. Stable Diffusion is a heavy AI model and needs GPU to render images quickly. On CPUs, it is very slow. On the other hand, GPUs are very costly on a Cloud platform. ## What if you could get a ready to use Stable Diffusion app on GPU servers at very low cost? > 🥳 **Introducing:** Dedicated Stable Diffusion GPU Instance on Q Blocks When we say dedicated, it means you can launch a GPU powered instance of Stable Diffusion dedicated to your account on Q Blocks at no extra cost. This results in faster renders of high quality images while being ultra low cost. > This is a No Code Solution. Thus, you don’t need any coding skills to launch Stable diffusion GPU server on Q Blocks. To get started simply follow this 30 second video or follow these 3 steps: ### 3 Steps to run Stable Diffusion Instance on Q Blocks: 1. Login to Q Blocks [dashboard](https://www.qblocks.cloud/client/v2/create-instance). 2. Select a GPU server with Stable Diffusion Image. 3. Launch the Instance and wait for a couple of minutes to get access. Then click on “select an action” dropdown and click on “Connect” button to access Stable diffusion interface. That’s it! > Once you are in the Stable Diffusion Web Interface simply click on “Generate” button to load the model. After the model is loaded, you can enter your text prompt in Text Area and Generate realistic images with Stable Diffusion Model. {% embed url="" %} {% hint style="danger" %} Don’t forget to “Destroy” your instance from dashboard after you are done with using Stable Diffusion application as your account will be charged for the time the instance is running. {% endhint %} ### Possibilities with Stable Diffusion: With stable diffusion model you can create any desired image. We are only limited by our imagination. As the field of Diffusion AI models progresses we will see more high fidelity and high quality outputs in coming time. For now, checkout some of the amazing images made with Stable Diffusion on Q Blocks: ![](/files/FGio7bWNZYHllAC3Gbv1) ![](/files/vrCZSlZosY2Pdum2lBA3) ![](/files/mfpnde1ErX899rXmce0w) ![](/files/1Ir5QzeOsGZ3mRkDi0RC) ![](/files/l3H8Moo8MUlvEctDUKeC) ## That’s all Folks! Now show us your creativity! 🚀 # Setup Docker with Nvidia GPU support {% hint style="info" %} Setup time less than 5 minutes {% endhint %} {% hint style="warning" %} Docker containers are currently supported on Data Center nodes only. On Consumer nodes, docker containers are supported only when rented on a monthly basis. {% endhint %} Docker is a tool designed to make it easier to create, deploy, and run applications by using containers. To run docker containerized applications inside your Q Blocks instance you'd need the docker engine to be installed correctly with Nvidia GPU support. ## How to install Docker engine with GPU support? Follow the steps below for successful installation: 1. Go to your Q Blocks instance and open the bash terminal in it. 2. Run the following command: ```bash wget 'https://www.qblocks.cloud/resources/setup-docker-gpu.sh' --no-check-certificate -O - | bash ``` It should take 2-3 minutes of installation time and should return the following output: "SUCCESSFULLY INSTALLED DOCKER WITH GPU SUPPORT." ### How to use Docker with GPUs? Once you have installed docker engine with GPU support in your Q Blocks instance, running a GPU enabled docker container is fairly straightforward. For example, if you have 4 GPUs in your instance and you plan to use the first 2 GPUs out of them for a docker Image then simply run a command like this in your instance: ```bash docker run --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=0,1 -it DOCKER_IMAGE ``` # Enable port forwarding on a Docker container in Q Blocks instance There could be a scenario where you want to publicly access the running docker container application in your instance. Docker containers support port forwarding through which you can access the application running inside your container at a specified port through browser or APIs. Q Blocks instances come with an open port by default that can be used to make an application running inside it publicly accessible. This example assumes that docker engine is already installed in your Q Blocks instance, if not then we would recommend to first install it via this [guide](https://www.notion.so/Setup-Docker-with-Nvidia-GPU-support-08e4a555ec6b461fa38d1242fdbd15a0?pvs=21). For this example we will use Nagios application docker Image: ```bash docker pull jasonrivers/nagios ``` Nagios is a monitoring tool and this image is used to launch a container with the Nagios application running inside the container at Port 80. Now to make it viewable on browser we will use port forwarding on the docker container run command. To enable port forwarding on a container running inside your Q Blocks instance, follow the steps below: 1. Click on the "More info" button for your instance on the Instances page in your Dashboard. 2. Copy the **Extra Port** and **Host URL** as shown in the image:
3. Run the docker container command as follows: ```bash docker run --name nagios4 -p EXTRA_PORT:80 -d jasonrivers/nagios:latest ``` 4. Now access the container application in your browser at the following URL: ```bash http://HOST_URL:EXTRA_PORT ``` This should help you get started with running port forwarded containerized applications that can be accessible publicly. {% hint style="danger" %} **Word of Caution:** Any application running at a forwarded port is accessible to the public. Right precautions with auth token or password access must be implemented to secure the applications. {% endhint %} If you still face issues setting it up then don't hesitate to reach out to us at: [support@qblocks.cloud](mailto:support@qblocks.cloud?subject=Unable%20to%20launch%20TensorBoard) # Run production ready lightweight kubernetes using K3s in Q Blocks instance K3s is a production-ready lightweight Kubernetes distribution that allows easy and scalable container orchestration. Read more on [K3s official Github Repo](https://github.com/k3s-io/k3s). ### Pre-requisites: * You need a pro / business Q Blocks instance * Ask [Q Blocks support](mailto:support@qblocks.cloud) to enable K3s support on your instance Once pre-requisite is fulfilled, we can proceed ahead with K3s setup. ### Steps to bring up K3s cluster inside Q Blocks GPU instance: 1. **Make sure `nvidia-smi` is running inside the container** 2. **Install Docker** ```bash sudo apt-get update sudo apt-get install docker.io ``` 3. **Install nvidia-container-toolkit** ```bash distribution=$(. /etc/os-release;echo $ID$VERSION_ID) curl -s -L https://nvidia.github.io/libnvidia-container/gpgkey | sudo apt-key add - curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | sudo tee /etc/apt/sources.list.d/libnvidia-container.list sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit ``` 4. **Set nvidia runtime as default container runtime:** By default, k3s prefers containerd runtime. But for GPUs to work we need default runtime of nvidia. So we setup nvidia runtime as follows in docker daemon file: ```bash sudo vim /etc/docker/daemon.json ``` ```json { "default-runtime": "nvidia", "runtimes": { "nvidia": { "path": "/usr/bin/nvidia-container-runtime", "runtimeArgs": [] } } } ``` ```bash sudo systemctl restart docker ``` 5. **Now, we will run setup K3s cluster using docker runtime:** First, we install K3s: ```bash sudo curl -sfL https://get.k3s.io | sh -s - --docker ``` 6. **Make sure k3s cluster is up and running** Wait for 5-10 seconds for the cluster to come up and then run this command: ```bash sudo k3s kubectl get pods --all-namespaces ``` 7. **Install NVIDIA daemon for K3s**\*: This makes instance GPU available for k3s cluster ```bash sudo k3s kubectl create -f https://raw.githubusercontent.com/NVIDIA/k8s-device-plugin/v0.14.1/nvidia-device-plugin.yml ``` **8. Do check logs of nvidia-device-plugin to confirm GPU are detected:** Get the name of nvidia pod launched by step 7 using this command's output: ```bash sudo k3s kubectl get pods --all-namespaces ``` Add the pod name in below command: ```bash sudo k3s kubectl logs -n kube-system ``` This should return an output like this: ```bash 0812 05:23:47.267089 1 main.go:154] Starting FS watcher. I0812 05:23:47.267213 1 main.go:161] Starting OS watcher. I0812 05:23:47.267548 1 main.go:176] Starting Plugins. I0812 05:23:47.267563 1 main.go:234] Loading configuration. I0812 05:23:47.267689 1 main.go:242] Updating config with default resource matching patterns. I0812 05:23:47.267884 1 main.go:253] Running with config: { "version": "v1", "flags": { "migStrategy": "none", "failOnInitError": false, "nvidiaDriverRoot": "/", "gdsEnabled": false, "mofedEnabled": false, "plugin": { "passDeviceSpecs": false, "deviceListStrategy": [ "envvar" ], "deviceIDStrategy": "uuid", "cdiAnnotationPrefix": "cdi.k8s.io/", "nvidiaCTKPath": "/usr/bin/nvidia-ctk", "containerDriverRoot": "/driver-root" } }, "resources": { "gpus": [ { "pattern": "*", "name": "nvidia.com/gpu" } ] }, "sharing": { "timeSlicing": {} } } I0812 05:23:47.267893 1 main.go:256] Retreiving plugins. I0812 05:23:47.268313 1 factory.go:107] Detected NVML platform: found NVML library I0812 05:23:47.268378 1 factory.go:107] Detected non-Tegra platform: /sys/devices/soc0/family file not found I0812 05:23:47.279615 1 server.go:165] Starting GRPC server for 'nvidia.com/gpu' I0812 05:23:47.280859 1 server.go:117] Starting to serve 'nvidia.com/gpu' on /var/lib/kubelet/device-plugins/nvidia-gpu.sock I0812 05:23:47.283115 1 server.go:125] Registered device plugin for 'nvidia.com/gpu' with Kubelet ``` 9. **Validate if GPUs are getting detected by K3s cluster node:** ```bash sudo k3s kubectl describe node -A | grep nvidia ``` 10. **If you are able to see GPU recognised and deamonSet not throwing an error its time to do a test run and make sure a pod can access the GPU. Make sure to run this container only on a node with GPU.** > Make sure the docker image used for testing has same or lower cuda version as the one supported by nvidia driver in Instance. 1. Create a .yaml file `k3sgputest.yaml` ```bash apiVersion: v1 kind: Pod metadata: name: gpu-pod spec: restartPolicy: Never containers: - name: cuda-container image: nvcr.io/nvidia/k8s/cuda-sample:vectoradd-cuda11.2.1-ubuntu18.04 resources: limits: nvidia.com/gpu: 1 # requesting 1 GPU tolerations: - key: nvidia.com/gpu operator: Exists ``` 2. Run the gpu pod ```bash sudo k3s kubectl apply -f k3sgputest.yaml sudo k3s kubectl logs gpu-pod ``` 4. Please wait for 5-10 seconds for the pod to load and run. If it ran successfully, it would display a log like this: ```bash [Vector addition of 50000 elements] Copy input data from the host memory to the CUDA device CUDA kernel launch with 196 blocks of 256 threads Copy output data from the CUDA device to the host memory Test PASSED Done ``` {% hint style="success" %} This confirms K3s cluster was able to detect GPU and pods are able to run code on GPUs inside kubernetes cluster {% endhint %} If you face any difficulty in setting up K3s then please reach us out at . # Upgrade CUDA to v12.2 This guide helps you upgrade your Q Blocks GPU instance CUDA version to 12.2. {% hint style="warning" %} CUDA 12 requires Nvidia Driver 535 or higher. Please confirm that by running command `nvidia-smi` in your Q Blocks instance. {% endhint %} **Uninstall any existing CUDA** ```bash sudo apt-get --purge remove -y '*cublas*' 'cuda*' 'nsight*' ``` ```bash sudo apt-get --purge remove -y '*nvidia*' ``` ```bash sudo rm -rf /usr/local/cuda* ``` ```bash sudo apt -y autoremove ``` **Install CUDA v12.2** ```bash CUDAVERSION=12.2 wget https://developer.download.nvidia.com/compute/cuda/12.2.2/local_installers/cuda_12.2.2_535.104.05_linux.run chmod +x cuda_12.2.2_535.104.05_linux.run sudo ./cuda_12.2.2_535.104.05_linux.run --toolkit --silent ``` **Set CUDA path** ```bash echo 'export PATH=/usr/local/cuda-$CUDAVERSION/bin:$PATH' >> ~/.bashrc echo 'export LD_LIBRARY_PATH=/usr/local/cuda-$CUDAVERSION/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc source ~/.bashrc sudo ldconfig ``` **Confirm if CUDA is visible** ```bash # Confirm if cuda is visible: nvcc -V ```