Using the Tensorent Dashboard

1. Logging In

  1. Go to https://dashboard.tensorent.com (This is a placeholder URL)
  2. Enter your credentials.
  3. Access your dedicated cloud instance panel.

2. Customizing Your Instance

After logging in, you can customize your instance to meet your specific requirements. The customization options are typically presented in a form or a series of steps.

Once you've made your selections, you'll typically have a "Create Instance" or "Launch Instance" button to start provisioning your customized GPU instance.

3. SSH Access to Your Instance

Secure Shell (SSH) is the standard method for securely accessing your remote instance. You'll need an SSH client (like OpenSSH on Linux/macOS or PuTTY on Windows).

Troubleshooting SSH:

4. Using the Integrated Notebook

Tensorent provides a built-in notebook environment for interactive computing, similar to Jupyter Notebook or JupyterLab, but optimized for performance and accuracy.

Troubleshooting Notebooks:

Hosting AI Models and Applications

1. Deploying an AI Model

This section describes how to deploy a trained AI model for inference (making predictions).

  1. Prepare your model:
    • Save your model: Save your trained model in a suitable format. Common formats include:
      • TensorFlow: SavedModel format (`.pb`) or HDF5 (`.h5`)
      • PyTorch: TorchScript (`.pt`) or a state dictionary (`.pth`)
      • Scikit-learn: Joblib (`.joblib`) or Pickle (`.pkl`)
      Example (TensorFlow): 
      # Save as SavedModel
tf.saved_model.save(model, "path/to/saved_model")

# Save as HDF5
model.save("path/to/model.h5")
      Example (PyTorch): 
      # Save as TorchScript
traced_model = torch.jit.trace(model, example_input)
torch.jit.save(traced_model, "path/to/model.pt")

# Save state dictionary
torch.save(model.state_dict(), "path/to/model.pth")
    • Create a prediction script: Write a Python script (e.g., `predict.py`) that loads your model and handles prediction requests. This script will act as the interface between your model and incoming requests. It should:
      • Load the saved model.
      • Preprocess input data (resize images, tokenize text, etc.).
      • Run the model's prediction function.
      • Post-process the output (convert probabilities to class labels, etc.).
      • Return the prediction in a suitable format (e.g., JSON).
      Example (TensorFlow, using Flask for a simple web server): 
      # predict.py
from flask import Flask, request, jsonify
import tensorflow as tf
import numpy as np

# Load the model (replace with your actual path)
model = tf.saved_model.load("path/to/saved_model")

app = Flask(__name__)

@app.route("/predict", methods=["POST"])
def predict():
    # Get data from request (assuming JSON with an 'image' key)
    data = request.get_json()
    image = np.array(data['image'])

    # Preprocess the image (example: resize)
    image = tf.image.resize(image, [224, 224])

    # Make a prediction
    predictions = model(tf.constant([image])) # Add batch dimension

    # Post-process (example: get the class with highest probability)
    predicted_class = np.argmax(predictions.numpy())

    # Return the result as JSON
    return jsonify({"predicted_class": int(predicted_class)})

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)  # Listen on all interfaces, port 5000
    • Create a requirements.txt file: List all the Python packages your prediction script depends on. This ensures that the correct environment can be created for your model. 
      # requirements.txt
flask
tensorflow
numpy
  2. Upload your files: Upload your model files, prediction script (`predict.py`), and `requirements.txt` file to your instance (using SCP, SFTP, or the notebook interface's file upload feature).
  3. Choose a runtime: Select the appropriate runtime environment (e.g., Python 3.8 with TensorFlow 2.x) through your Tensorent dashboard.
  4. Set resource limits: Specify the CPU, memory, and GPU resources allocated to your deployment. This prevents your model from consuming excessive resources and affecting other services.
  5. Deploy the model: Initiate the deployment process through your dashboard. This typically involves:
    • Creating a container (e.g., using Docker) that packages your model, script, and dependencies.
    • Starting the container on your instance.
    • Setting up a network endpoint (e.g., a URL with a port) that allows external access to your model.
  6. Obtain the API endpoint: Once deployed, the dashboard will provide you with an API endpoint (a URL) that you can use to send prediction requests to your model. This endpoint might look like: `http://your-instance-ip:5000/predict`.
  7. Test your deployment: Send a test request to your API endpoint to verify that it's working correctly. You can use tools like `curl` (command-line) or Postman (GUI) to send requests. Example (`curl`): 
    curl -X POST -H "Content-Type: application/json" -d '{"image": [[...image data...]]}' http://your-instance-ip:5000/predict
    Replace `[...image data...]` with the actual image data in the format expected by your `predict.py` script.

2. Hosting Games and Applications

Hosting games and other applications is conceptually similar to deploying AI models, but the specific steps may vary depending on the application.

  1. Prepare your application: Ensure your game or application is configured to run in a server environment. This might involve:
    • Configuring network settings (ports, IP addresses).
    • Setting up any required databases or external services.
    • Creating configuration files.
  2. Containerize (Recommended): Package your application and its dependencies into a container (e.g., using Docker). This makes deployment more reliable and portable. 
    # Example Dockerfile
FROM ubuntu:latest

# Install dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    python3 python3-pip 

# Copy your application files
WORKDIR /app
COPY . .

# Install Python requirements
RUN pip3 install -r requirements.txt

# Expose the necessary port (replace 8080 with your application's port)
EXPOSE 8080

# Run your application
CMD ["python3", "app.py"]
  3. Upload: Upload your application files or Docker image to your instance.
  4. Define server specs: Specify the CPU, memory, and GPU resources required by your application.
  5. Network and security:
    • Configure firewall rules to allow traffic to the appropriate ports.
    • Consider setting up a domain name and SSL certificate for secure access (especially for web applications).
  6. Launch: Start your application or container through the dashboard.
  7. Monitor: Use the dashboard to monitor the resource usage and performance of your application.

API Reference

Tensorent provides a RESTful API for managing your instances programmatically. All API requests should be made over HTTPS. The base URL for the API is: `https://api.tensorent.com/v1` (This is a placeholder URL; use the actual URL provided by Tensorent).

Authentication

API requests are authenticated using API keys. You can generate API keys from your Tensorent dashboard. Include your API key in the `Authorization` header of each request:

Authorization: Bearer YOUR_API_KEY

Replace `YOUR_API_KEY` with your actual API key. Keep your API key secret and do not share it.

Instance Management

These endpoints allow you to create, manage, and delete GPU instances.

Resource Monitoring

These endpoints provide real-time metrics on resource usage.

Snapshots

Snapshots allow you to create backups of your instances.

Automated Scaling

This section is a placeholder. Automated scaling often requires custom setup and integration with monitoring services. Contact Tensorent support for details.

Possible API endpoints (Illustrative):

Error Handling

The API uses standard HTTP status codes to indicate success or failure. Common error codes include:

Error responses typically include a JSON body with more details:

{
  "error": {
    "code": "invalid_request",
    "message": "The 'gpu_type' parameter is required."
  }
}

Tensorent – Powering the Future of AI and Cloud Computing.