73 lines
3.9 KiB
Markdown
73 lines
3.9 KiB
Markdown
# Monitoring-Database_686083870
|
|
## Introduction
|
|
|
|
This page presents all the monitoring used to cover database level monitoring, not limited to DB instance level, application level monitoring.
|
|
|
|
## Detailed monitors
|
|
|
|
#### Instance level monitoring
|
|
|
|
- Locks - blocked sessions(TBD) - check every 1 minute
|
|
- Graph (To be reviewed)
|
|
```
|
|
SELECT count(bl.pid) AS blocked_pid_count FROM pg_catalog.pg_locks bl JOIN pg_catalog.pg_stat_activity a ON a.pid = bl.pid JOIN pg_catalog.pg_locks kl ON kl.transactionid = bl.transactionid AND kl.pid != bl.pid JOIN pg_catalog.pg_stat_activity ka ON ka.pid = kl.pid WHERE NOT bl.granted and a.query_start <= (now() - interval '5 minutes');
|
|
```
|
|
- Table
|
|
```
|
|
SELECT bl.pid AS blocked_pid, a.usename AS blocked_user, a.query_start AS blocked_start_time , a.query AS blocked_statement, now() - a.query_start AS blocked_duration,ka.query AS blocking_statement, ka.query_start AS blocking_start_time , now() - ka.query_start AS blocking_duration, kl.pid AS blocking_pid, ka.usename AS blocking_user FROM pg_catalog.pg_locks bl JOIN pg_catalog.pg_stat_activity a ON a.pid = bl.pid JOIN pg_catalog.pg_locks kl ON kl.transactionid = bl.transactionid AND kl.pid != bl.pid JOIN pg_catalog.pg_stat_activity ka ON ka.pid = kl.pid WHERE NOT bl.granted and a.query_start <= (now() - interval '5 minutes');
|
|
```
|
|
- Long active queries (TBD) - check every 10 minutes
|
|
- Graph
|
|
```
|
|
SELECT count(pid) AS Long_Query_count FROM pg_catalog.pg_stat_activity WHERE (now() - pg_catalog.pg_stat_activity.query_start) > interval '30 minutes' and state = 'active';
|
|
```
|
|
- Table
|
|
```
|
|
SELECT pid, now() - pg_catalog.pg_stat_activity.query_start AS duration, state FROM pg_catalog.pg_stat_activity WHERE (now() - pg_catalog.pg_stat_activity.query_start) > interval '30 minutes' and state = 'active';
|
|
```
|
|
- Capture RDS top 10 query (TBD) - check every 10 minutes
|
|
1. Clean stat\_statement bi-weekly
|
|
2. capture during runtime if CPU is more than 97% for 60 mins
|
|
3. Table
|
|
```
|
|
Select (select datname from pg_database where oid=dbid) as db_name, query as short_query, round(total_time::numeric, 2) as total_time, calls, round(mean_time::numeric, 2) as mean, round((100 * total_time / sum(total_time::numeric) over ())::numeric, 2) as percentage_overall from pg_stat_statements order by total_time desc limit 20;
|
|
```
|
|
4. Graph (Optional)
|
|
```
|
|
Select (select datname from pg_database where oid=dbid) as db_name, query as short_query, round((100 * total_time / sum(total_time::numeric) over ())::numeric, 2) as percentage_overall from pg_stat_statements order by total_time desc limit 20;
|
|
```
|
|
- Dead tuple - logical database level (TBD) - check every 10 minutes
|
|
- Table
|
|
```
|
|
-- Top bloating tables
|
|
select relname, n_dead_tup, last_vacuum, last_autovacuum from
|
|
pg_catalog.pg_stat_all_tables
|
|
where n_dead_tup > 0 order by n_dead_tup desc limit 20;
|
|
```
|
|
- Graph (Optional)
|
|
```
|
|
-- Top bloating tables
|
|
select relname, n_dead_tup
|
|
pg_catalog.pg_stat_all_tables
|
|
where n_dead_tup > 0 order by n_dead_tup desc limit 20;
|
|
```
|
|
|
|
#### Database level customer metrics
|
|
|
|
1. NativeSACM Transaction Context Queue - check every 10 minutes
|
|
```
|
|
select count(*) from transaction_context_$tenant where entity::jsonb->>'entity_type' in('Device','SystemElement','ServiceComponent','ActualService') ;
|
|
```
|
|
2. NativeSACM Transaction Context Queue retries - check every 10 minutes
|
|
```
|
|
select count(*) from transaction_context_$tenant where retry_count=2 and entity::jsonb->>'entity_type' in('Device','SystemElement','ServiceComponent','ActualService') ;
|
|
```
|
|
3. SLT Job queue - check every 10 minutes
|
|
```
|
|
select count(1) from transaction_context_$tenant a, transaction_etl_job_$tenant b where a.transaction_timestamp > b.last_taken and a.flag0 & 1 = 1 and b.job_name = 'SLT';
|
|
```
|
|
4. SLT Delay time (This may need to be removed if it's the same as SLT Job Queue) - check every 10 minutes
|
|
```
|
|
SELECT now() - last_taken as delay FROM maas_admin.transaction_etl_job_$tenant where job_name = 'SLT';"
|
|
```
|