Agent Setup
OtterTune provides an open source agent capable of observing database and CloudWatch metrics that runs on your hosts. This allows OtterTune to optimize and monitor your database without needing direct access.
The source code for OtterTune's agent is available on GitHub: https://github.com/ottertune/ot-agent
When you add a database to OtterTune, you will be prompted to add an agent via the Agent Setup page.
CloudFormation
Docker
Kubernetes
Helm
OtterTune will provide an external link to create a Fargate Container via AWS CloudFormation. Some fields will be prefilled for you, while others you will need to fill in.
You will need to supply the SecurityGroupIDs and SubnetIDs parameters such that the specified security groups and subnets allow:
- 1.The container to connect to your database
- 2.Outbound https traffic to 0.0.0.0/0 so the container can push data to OtterTune.
OtterTune provides a prebuilt Docker container on DockerHub. A simple
docker run command is provided on the Agent Setup page. Some values will be prefilled for you, while others need to be filled in. See Agent Information below for more details.
OtterTune provides a Kubernetes manifest to deploy the OtterTune Agent to your cluster. The Agent Setup page shows a copy of the manifest for you to load into your favorite text editor. The manifest has a few environment variables marked as
<REDACTED>; you will need to fill in these environment variables with your own values before deploying the manifest. The sections Agent Information and Additional Environment Variables below describe each of these environment variables.Once you've replaced all occurrences of
<REDACTED> with real values, you can deploy the manifest to your cluster with kubectl apply -f /path/to/manifest.yml, where /path/to/manifest is the path to the copy of the manifest.Omitting AWS Credentials: It is possible to omit your AWS credentials from the deployment manifest. AWS offers a mechanism to attach IAM roles to Kubernetes
ServiceAccounts. This allows the OtterTune Agent to connect to AWS RDS, eliminating the need to inject credentials into the Agent's container via environment variables. To do this, you must edit the deployment manifest to use a Kubernetes ServiceAccount with the appropriate IAM role.For example, if you have an IAM Role
OTAgentRole, and the ARN is arn:aws:iam::123456789:role/OTAgentRole, then you might have a ServiceAccount like this:apiVersion: v1
kind: ServiceAccount
metadata:
name: OTAgentServiceAccount
annotations:
eks.amazonaws.com/role-arn: arn:aws:iam::123456789:role/OTAgentRole
which you can also deploy with
kubectl apply -f /path/to/sa/manifest.yml You could then alter the manifest to attach the ServiceAccount like so:apiVersion: apps/v1
kind: Deployment
metadata:
name: ottertune-agent
spec:
selector:
matchLabels:
app.kubernetes.io/name: "OtterTune-Agent"
replicas: 1
template:
metadata:
labels:
app.kubernetes.io/name: "OtterTune-Agent"
spec:
serviceAccountName: OTAgentServiceAccount
containers:
- name: ottertune-agent
imagePullPolicy: IfNotPresent
image: public.ecr.aws/ottertune/agent:0.3.3
env:
- name: "AWS_REGION"
value: "${region}"
- name: "OTTERTUNE_DB_IDENTIFIER"
value: "<REDACTED>"
- name: "OTTERTUNE_DB_USERNAME"
value: "<REDACTED>"
- name: "OTTERTUNE_DB_PASSWORD"
value: "<REDACTED>"
- name: "OTTERTUNE_API_KEY"
value: "<REDACTED>"
- name: "OTTERTUNE_DB_KEY"
value: "<REDACTED>"
- name: "OTTERTUNE_ORG_ID"
value: "<REDACTED>"`
OtterTune provides a Helm chart to deploy the OtterTune agent to your cluster. For quick setup please read on. For more advanced configuration, please go to agent-helm-chart.
To install,
To get started, first add the Helm repo with your CLI:
helm repo add ottertune https://ottertune.github.io/agent-helm-chart/
Next, refresh your local Helm index.
helm repo update
Finally, create a YAML file with your values. The OtterTune agent setup UI will provide some of the values listed below for you to fill in. OtterTune will redact certain values that you will need to fill in yourself (such as database username and password).
# my-values.yaml
aws:
region: "us-east-2"
accessKeyID: "<FAKE_ACCESS_ID>"
secretAccessKey: "<FAKE_SECRET_KEY>"
ottertune:
postgresDBName: "my-database-name" # Only for Postgres Databases
dbIdentifier: "my-database"
dbUsername: "example-username"
dbPassword: "example-password"
apiKey: "example-api-key"
dbKey: "example-db-uuid"
orgID: "example-org-uuid"
Then install the chart
helm install -f my-values.yaml my-release-name ottertune/ottertune-agent
For more configuration options such as ways to leverage kubernetes secrets to store the OtterTune API Key and ways to integrate IAM roles directly with the agent (as opposed to using IAM user access/secret keys), please see our public repo agent-helm-chart.
OtterTune will generate a secret API Key that can be used to authenticate Agent information being passed to the OtterTune service. This value can be retrieved by navigating to the Agent Setup page for any of your databases.
This API Key is a Secret value. Do not share it publicly.
OTTERTUNE_DB_IDENTIFIER | Amazon Web Services (AWS) db identifier for your database in Relational Database Service (RDS). |
OTTERTUNE_DB_USERNAME | Username within DBMS software for OtterTune Agent to act as. |
OTTERTUNE_DB_PASSWORD | Password for OtterTune to sign into the DBMS. |
OTTERTUNE_API_KEY | Secret value used to authenticate your organization's agents with the OtterTune service. |
AWS_REGION | AWS region your DBMS is located in (eg: us-east-2) |
OTTERTUNE_DB_KEY | Identifier for the DBMS this agent is collecting information from. Should be autofilled by OtterTune. |
OTTERTUNE_ORG_ID | Public identifier for your organization. Should be autofilled by OtterTune. |
POSTGRES_OTTERTUNE_DB_NAME | Specific database(s) in DBMS to collect metrics from. Only needed by agents connecting to a PostgreSQL database. For multiple databases, enter a comma separated list. OtterTune collects table and index statistics in those databases. Note: the first database in the list is considered the primary database. |
Note: You can skip this section if you deployed your agent via CloudFormation since it has already set up the proper policies and roles.
This section still applies if you created your account-wide IAM role via CloudFormation in the Create IAM Role step but wish to deploy your agent via another method.
The OtterTune Agent needs valid AWS credentials for a user or role to inspect the database and collect CloudWatch metrics for the database you wish to optimize. We recommend one of the following approaches:
Ensure your environment (e.g., Kubernetes pod, Fargate task, EC2 instance profile) is running as an IAM role with the appropriate AWS permissions.
In addition to the environment variables mentioned in the Agent Information section, please provide the AWS access and AWS secret access keys for a user with the appropriate AWS permissions.
Environment Variable | Description |
|---|---|
AWS_ACCESS_KEY_ID | Access key of IAM user |
AWS_SECRET_ACCESS_KEY | Secret access key of IAM user |
The AWS role or user should contain at least the following permissions :
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"rds:Describe*",
"cloudwatch:Get*",
],
"Resource": [
"*"
],
"Effect": "Allow"
},
]
}
See possible Agent Statuses below for more information.
Uninitialized
Disconnected
Connected
Loading
Unknown
OtterTune has never seen an agent successfully report monitoring data for this database. Double-check your agent setup. If you would like help resolving the issue, reach out to [email protected] or contact us on Slack.
OtterTune has never seen an Agent report observations successfully for this database. Double-check your agent setup. If you would like help resolving the issue, reach out to [email protected] or contact us on Slack.
OtterTune has seen your agent report information successfully. No action is needed.
Note: OtterTune considers an agent to be connected if it has reported data in the last 5 minutes, so expect a delay when you disable an agent.
OtterTune is currently determining the agent status. Remain on the page to see the status.
OtterTune was not able to determine your current agent status. Try reloading the page. If you would like help resolving the issue, reach out to [email protected] or contact us on Slack.
To upgrade your agent to the latest version, please follow the steps below.
Fargate via CloudFormation
Docker
First, please find out the latest version number from this page.
Then go to the CloudFormation Stack you created for the agent. Click Update at the right top.
Step 1, choose Use current template, click Next.
Step 2, edit Image URL with the current agent version number.
Then click Next.
Step 3, click Next.
Finally, click Update stack to finish the update.
You need to delete the old agent Docker container. Then run the command on the Agent Setup page to create a new Docker container. Some values will be prefilled for you, while others need to be filled in. The agent version in the command will be updated to the latest one.
To collect all relevant monitoring data, you will need to make sure the user your agent uses to connect to your database has been granted the following permissions. Note if you use a user other than
ottertune you should alter the below commands accordingly.MySQL
PostgreSQL
GRANT PROCESS ON *.* TO 'ottertune';
GRANT REPLICATION CLIENT ON *.* TO 'ottertune';
GRANT SHOW VIEW ON *.* TO 'ottertune';
GRANT SELECT ON mysql.innodb_index_stats TO 'ottertune';
GRANT SELECT ON performance_schema.table_io_waits_summary_by_index_usage TO 'ottertune';
GRANT SELECT ON performance_schema.events_statements_summary_by_digest TO 'ottertune';
-- if mysql version >= 8.0
GRANT SELECT ON performance_schema.events_statements_histogram_global TO 'ottertune';
GRANT pg_monitor TO ottertune;
# Only required in the primary database if you specified multiple databases.
Last modified 1mo ago