1024programmer Java MongoDB Database Analyzer

MongoDB Database Analyzer

Database | mysql tutorialMongoDB Database Analyzer MongoDB , database, analyzer, introduction, database, running, slow time, my database-mysql tutorial to set up the member’s website source code, ubuntu uses web page download, Zhihu python crawler, php multi-he, google promotion seolzwA, introduction when the database is running slow , we need to find out the performance bottleneck. The first step in identifying performance bottlenecks is to find out what the database is busy with, and then identify the slowest operations among them to optimize. MongoDB provides a database analyzer to collect detailed information about write operations, cursors, commands, etc. executed on the instance. MongoDB database analyzer can be used in instance-level creative h5 source code, ubuntu configuration in vmw, tomcat why tcp6, py crawler advantages, Christmas hat avatar generation source code php, Conghua fast seolzw table query source code, ubuntu installation GPU driver, how to change tomcat’s ip, Crawler analysis website password, PHP big data processing analysis solution, Zhuhai Yahoo seolzwA, Introduction

When the database runs slowly, we need to find out the performance bottlenecks. The first step in identifying performance bottlenecks is to find out what the database is busy with, and then identify the slowest operations among them to optimize. MongoDB provides a database analyzer to collect detailed information about write operations, cursors, commands, etc. executed on the instance. The MongoDB database analyzer can be opened at the instance level or at the database level.
The MongoDB database analyzer can be set to three levels:
0 – The analyzer is turned off and no data is collected
1 – Only analysis data for slower operations is collected. By default, if an operation takes more than 100ms, it is considered a slower operation.
2 – Collect analytics data for all database operations.

B. system.profile collection

The data collected by the MongoDB database analyzer will be stored in the system.profile collection. system.profile is a fixed-size collection into which data will be written in a loop. When all allocated space is used up, MongoDB will overwrite the oldest document in the collection. By default, the system.profile collection size is 4M.

Commonly used fields in the system.profile collection include:
ts: timestamp when the database operation occurs
op: operation type, possible values ​​are insert, query, update, remove, getmore,command
ns: the name of the operation object
query: query statement
nreturned: the number of documents returned
nscanned: the number of documents scanned to perform the operation
millis: Time spent performing operations

C. Setting the database profiler level

The MongoDB database profiler has three levels, which can be set through the mongo Shell:

db.getProfilingLevel (): Get the current profile level

db.getProfilingStatus(): Get the current profile level and return the slowms threshold.

db.setProfilingLevel(): Set profile level

We can try to set the analyzer level to 1 and slowms to 500ms:

db. setProfilingLevel(1,500)
{ “was” : 0, “slowms” : 100, “ok” : 1 }

The was field in the execution result shows that the original analyzer level is 0, and the slowms field shows The original slow operation threshold is 100ms. Make db.getProfilingStatus() query the new profiler settings:

db.getProfilingStatus()
{ “was” : 1, “slowms” : 500 }

Execute The results show that the analyzer level has been set to 1, indicating that mongodb will log all operations that take more than 500ms to execute.

D. View the analyzer data and analyze it

First we prepare the initial data, establish the mydb library, build a test collection in it, and insert 10 million pieces of data:

p>

use mydb

db.test.drop()

for (var i=1;i<=10000000;i++) db.test.insert({empno :i,name:i,address:'AAAAAAAAAAAAAAAAAAAAAAAAAAA'})

Clear the system.profile collection, set the analysis level to 2, and the slow operation threshold to 100ms:
use mydb
db.setProfilingLevel(0)
db.system.profile.drop()
db.setProfilingLevel(2,100)

Query empno greater than without index Documents for 99999999:

db.test.find({empno:{$gt:9999999}})

A result is found, so the network:
{ “_id ” : ObjectId(“523ad4d465fdfe550ae05d97”), “empno” : 10000000, “name” : 10000000, “address” : “AAAAAAAAAAAAAAAAAAAAAAAAAAA” }

Create index in empno:

db. test.ensureIndex({empno:1})

Requerying documents with empno greater than 99999999 yields the same result.

Query the analysis data stored in system.profile and sort it according to the time of occurrence:

db.system.profile.find().limit(10).sort( { ts : 1 } ).pretty()

Get the result:
{
“op” : “query”,
“ns” : “mydb.system.indexes”,
“query” : {
“expireAfterSeconds” : {
“$exists” : true
}
},
“ntoreturn” : 0,
“ntoskip” : 0,
“nscanned” : 1,
“keyUpdates” : 0,
“numYield” : 0,
“lockStats” : {
/>”timeLockedMicros” : {
“r” : NumberLong(44),
“w” : NumberLong(0)
},
“timeAcquiringMicros” : {
“r” : NumberLong(2),
“w” : NumberLong(57806)
}
},
“nreturned” : 0,
“responseLength” : 20,
“millis” : 0,
“ts” : ISODate(“2013-09-19T10:47:00.656Z”),
“client” : “0.0.0.0”,
“allUsers” : [
{
“user” : “__system”,
“userSource” : “local”
}
],
“user ” : “__system@local”
}
{
“op” : “query”,
“ns” : “mydb.test”,
“query” : {
“empno” : {
“$gt” : 9999999
}
},
“ntoreturn” : 0,
“ntoskip” : 0,
“nscanned” : 10000000,
“keyUpdates” : 0,
“numYield” : 29,
“lockStats” : {
“timeLockedMicros” : {
“r” : NumberLong(59258536),
“w” : NumberLong(0)
},
“timeAcquiringMicros” : {
“r” : NumberLong( 29469480),
“w” : NumberLong(6)
}
},
“nreturned” : 1,
“responseLength” : 112,
“millis” : 29861,
“ts” : ISODate(“2013-09-19T10:47:21.476Z”),
“client” : “127.0.0.1”,
“allUsers ” : [ ],
“user” : “”
}
{
“op” : “query”,
“ns” : “mydb.system. indexes”,
“query” : {
“expireAfterSeconds” : {
“$exists” : true
}
},
“ntoreturn” : 0,
“ntoskip” : 0,
“nscanned” : 1,
“keyUpdates” : 0,
“numYield” : 0,
“lockStats” : {
“timeLockedMicros” : {
“r” : NumberLong(171),
“w” : NumberLong(0)
},
“timeAcquiringMicros” : {
“r” : NumberLong(8),
“w” : NumberLong(4)
}
},
“nreturned” : 0,
“responseLength” : 20,
“millis” : 0,
“ts” : ISODate(“2013-09-19T10:48:00.661Z”),
“client” : “0.0 .0.0″,
“allUsers” : [
{
“user” : “__system”,
“userSource” : “local”
}
],
“user” : “__system@local”
}
{
“op” : “query”,
“ns” : “mydb.system. indexes”,
“query” : {
“expireAfterSeconds” : {
“$exists” : true
}
},
“ntoreturn” : 0,
“ntoskip” : 0,
“nscanned” : 1,
“keyUpdates” : 0,
“numYield” : 0,
“lockStats” : {
“timeLockedMicros” : {
“r” : NumberLong(41),
“w” : NumberLong(0)
},
“timeAcquiringMicros” : {
“r” : NumberLong(3),
“w” : NumberLong(2)
}
},
“nreturned” : 0,
“responseLength” : 20,
“millis” : 0,
“ts” : ISODate(“2013-09-19T10:49:00.677Z”),
“client” : “0.0 .0.0″,
“allUsers” : [
{
“user” : “__system”,
“userSource” : “local”
}
],
“user” : “__system@local”
}
{
“op” : “insert”,
“ns” : “mydb.system. indexes”,
“query” : {
“v” : 1,
“key” : {
“empno” : 1
},
“ns” : “mydb.test”,
“name” : “empno_1”
},
“ninserted” : 1,
“keyUpdates” : 0,
“numYield” : 0,
“lockStats” : {
“timeLockedMicros” : {
“r” : NumberLong(0),
“w” : NumberLong(145940925)
},
“timeAcquiringMicros” : {
“r” : NumberLong(0),
“w” : NumberLong(292871)
}
} ,
“millis” : 145977,
“ts” : ISODate(“2013-09-19T10:51:37.246Z”),
“client” : “127.0.0.1”,
“allUsers” : [ ],
“user” : “”
}
{
“op” : “query”,
“ns” : “mydb.system.indexes”,
“query” : {
“expireAfterSeconds” : {
“$exists” : true
}
},
“ntoreturn” : 0,
“ntoskip” : 0,
“nscanned” : 2,
“keyUpdates” : 0,
“numYield” : 1,
“lockStats” : {
“timeLockedMicros” : {
“r” : NumberLong(598801),
“w” : NumberLong(0)
},
“timeAcquiringMicros” : {
“r” : NumberLong(96316198),
“w” : NumberLong(13)
}
},
“nreturned” : 0 ,
“responseLength” : 20,
“millis” : 1107,
“ts” : ISODate(“2013-09-19T10:51:38.122Z”),
“ client” : “0.0.0.0”,
“allUsers” : [
{
“user” : “__system”,
“userSource” : “local”
}
],
“user” : “__system@local”
}
{
“op” : “query”,
“ns” : “mydb.system.indexes”,
“query” : {
“expireAfterSeconds” : {
“$exists” : true
}
},
“ntoreturn” : 0,
“ntoskip” : 0,
“nscanned” : 2,
“keyUpdates” : 0,
“numYield” : 0,
“lockStats” : {
“timeLockedMicros” : {
“r” : NumberLong(54),
“w” : NumberLong(0)
},
“timeAcquiringMicros” : {
“r” : NumberLong(3),
“w” : NumberLong(2)
}
},
“nreturned” : 0 ,
“responseLength” : 20,
“millis” : 0,
“ts” : ISODate(“2013-09-19T10:52:38.260Z”),
“ client” : “0.0.0.0”,
“allUsers” : [
{
“user” : “__system”,
“userSource” : “local”
}
],
“user” : “__system@local”
}
{
“op” : “query”,
“ns” : “mydb.test”,
“query” : {
“empno” : {
“$gt” : 9999999
}
},
“ ntoreturn” : 0,
“ntoskip” : 0,
“nscanned” : 1,
“keyUpdates” : 0,
“numYield” : 0,
” lockStats” : {
“timeLockedMicros” : {
“r” : NumberLong(125810),
“w” : NumberLong(0)
},
“timeAcquiringMicros ” : {
“r” : NumberLong(4),
“w” : NumberLong(6)
}
},
“nreturned” : 1,
“responseLength” : 112,
“millis” : 125,
“ts” : ISODate(“2013-09-19T10:52:40.258Z”),
“client” : “127.0.0.1”,
“allUsers” : [ ],
“user” : “”
}

We see the same query executed twice and an index creation operation are included.
The timestamp of the first query is 2013-09-19T10:47:21.476Z, nscanned is 10 million, nreturned is 1, and the operation itself takes 29861ms.

The above query was executed without an index. To optimize the query, we created an index on empno. From the system.profile query results, we can see that the op that creates the index is insert, the namespace of the operation is mydb.system.indexes, and the name of the created index is empno_1. Creating the index itself took 145977ms.

The timestamp of our second execution of the query is 2013-09-19T10:52:40.258Z, nscanned is 1, nreturned is 1, and the operation itself takes 125ms.

From the system.profile output, we can see that queries with indexes are much faster than queries without indexes.

“w” : NumberLong(0)
},
“timeAcquiringMicros” : {
“r” : NumberLong(96316198),
“w” : NumberLong(13)
/>}
},
“nreturned” : 0,
“responseLength” : 20,
“millis” : 1107,
“ts” : ISODate(“ 2013-09-19T10:51:38.122Z”),
“client” : “0.0.0.0”,
“allUsers” : [
{
“user” : ” __system”,
“userSource” : “local”
}
],
“user” : “__system@local”
}
{
“op” : “query”,
“ns” : “mydb.system.indexes”,
“query” : {
“expireAfterSeconds” : {
“$ exists” : true
}
},
“ntoreturn” : 0,
“ntoskip” : 0,
“nscanned” : 2,
” keyUpdates” : 0,
“numYield” : 0,
“lockStats” : {
“timeLockedMicros” : {
“r” : NumberLong(54),
“w” : NumberLong(0)
},
“timeAcquiringMicros” : {
“r” : NumberLong(3),
“w” : NumberLong(2)
/>}
},
“nreturned” : 0,
“responseLength” : 20,
“millis” : 0,
“ts” : ISODate(“ 2013-09-19T10:52:38.260Z”),
“client” : “0.0.0.0”,
“allUsers” : [
{
“user” : ” __system”,
“userSource” : “local”
}
],
“user” : “__system@local”
}
{
“op” : “query”,
“ns” : “mydb.test”,
“query” : {
“empno” : {
“$gt” : 9999999
}
},
“ntoreturn” : 0,
“ntoskip” : 0,
“nscanned” : 1,
“keyUpdates” : 0,
“numYield” : 0,
“lockStats” : {
“timeLockedMicros” : {
“r” : NumberLong(125810),
“w ” : NumberLong(0)
},
“timeAcquiringMicros” : {
“r” : NumberLong(4),
“w” : NumberLong(6)
}
},
“nreturned” : 1,
“responseLength” : 112,
“millis” : 125,
“ts” : ISODate(“2013- 09-19T10:52:40.258Z”),
“client” : “127.0.0.1”,
“allUsers” : [ ],
“user” : “”
}

We see two executions of the same query and an index creation operation.
The timestamp of the first query is 2013-09-19T10:47:21.476Z, nscanned is 10 million, nreturned is 1, and the operation itself takes 29861ms.

The above query was executed without an index. To optimize the query, we created an index on empno. From the system.profile query results, we can see that the op that creates the index is insert, the namespace of the operation is mydb.system.indexes, and the name of the created index is empno_1. Creating the index itself took 145977ms.

The timestamp of our second execution of the query is 2013-09-19T10:52:40.258Z, nscanned is 1, nreturned is 1, and the operation itself takes 125ms.

From the system.profile output, we can see that queries with indexes are much faster than queries without indexes.

This article is from the internet and does not represent1024programmerPosition, please indicate the source when reprinting:https://www.1024programmer.com/749437

author: admin

Previous article
Next article

Leave a Reply

Your email address will not be published. Required fields are marked *

Contact Us

Contact us

181-3619-1160

Online consultation: QQ交谈

E-mail: [email protected]

Working hours: Monday to Friday, 9:00-17:30, holidays off

Follow wechat
Scan wechat and follow us

Scan wechat and follow us

Follow Weibo
Back to top
首页
微信
电话
搜索