dump & analysis
description
/**
<pre>
The buffer layout for fixed data part is as follows:
+-----------------------------------+
| table_id | Reserved for future use|
+-----------------------------------+
</pre>
<pre>
The buffer layout for variable data part is as follows:
+--------------------------------------------------------------------------+
| db len| db name | table len| table name | no of cols | array of col types|
+--------------------------------------------------------------------------+
+---------------------------------------------+
| metadata len | metadata block | m_null_bits |
+---------------------------------------------+
</pre>
@param buf Contains the serialized event.
@param fde An FDE event (see Rotate_event constructor for more info).
*/
dump dump
=== TableMapEvent ===
Date: 2021-09-26T17:04:42
Log position: 1049
Event size: 56
Read bytes: 47
Table id: 43256
Schema: will
Table: test_table_map_ev
Columns: 6
=== TableMapEvent ===
Date: 2021-09-27T14:59:13
Log position: 1597
Event size: 35
Read bytes: 28
Table id: 43257
Schema: will
Table: bin
Columns: 3
dump packet
packet length: 80
call[1]: dump (line 224)
call[2]: <module> (line 1)
call[3]: _getval (line 1155)
call[4]: do_p (line 1177)
call[5]: onecmd (line 217)
call[6]: onecmd (line 418)
------------------------------------------------------------------
00 9A 29 50 61 13 01 00 00 00 4F 00 00 00 19 04 ..)Pa.....O.....
00 00 00 00|F8 A8 00 00 00 00|01 00|04|77 69 6C .............wil
6C|00|11|74 65 73 74 5F 74 61 62 6C 65 5F 6D 61 l..test_table_ma
70 5F 65 76|00|06|08 0F FC FC 11 04|06:28 00-02- p_ev.........(..
02-00-04|3E|01 01 00 02 03 2D 02 3F D2 6C 87 D5 ...>.....-.?.l..
------------------------------------------------------------------
packet length: 59
call[1]: dump (line 224)
call[2]: <module> (line 1)
call[3]: _getval (line 1155)
call[4]: do_p (line 1177)
call[5]: onecmd (line 217)
call[6]: onecmd (line 418)
------------------------------------------------------------------
00 B1 5D 51 61 13 01 00 00 00 3A 00 00 00 3D 06 ..]Qa.....:...=.
00 00 00 00 F9 A8 00 00 00 00 01 00 04|77 69 6C .............wil
6C|00|03|62 69 6E|00|03|08 FE 0F|04:FE 01-0A 00| l..bin..........
06|01 01 00 02 01 3F 2B A6 38 A2 ......?+.8.
------------------------------------------------------------------
parse packet
https://dev.mysql.com/doc/internals/en/com-query-response.html#packet-Protocol::MYSQL_TYPE_STRING 보고 pymysql / pymysqlreplication에 추가
field length | field name | field value |
---|---|---|
6 | table id | 43224 |
2 | flags | 1 |
string(4) | schema | will |
string(17) | table name | test_table_map_ev |
lenencint | column count | 06 |
column count | column-def | LONGLONG, VARCHAR, BLOB, BLOB, TIMESTAMP2, FLOAT |
lenenc-str(6: 0, 2, 1, 1, 1, 1) | column-meta-def | |
1 | NULL-bitmask | 3E = 111110 역순 |
field length | field name | field value |
---|---|---|
6 | table id | 43224 |
2 | flags | 1 |
string(4) | schema | will |
string(3) | table name | bin |
lenencint | column count | 03 |
column count | column-def | LONGLONG, STRING, VARCHAR |
lenenc-str(4: 0, 2, 2) | column-meta-def | |
1 | NULL-bitmask | 6 = 110 역순 |
parse reference
rows_event.cpp:Table_map_event::Table_map_event() log_event.cc:Table_map_log_event::Table_map_log_event()
meta length for each column type
https://dev.mysql.com/doc/internals/en/table-map-event.html (outdated) https://github.com/mysql/mysql-server/blob/beb865a960b9a8a16cf999c323e46c5b0c67f21f/sql/log_event.cc#L11305-L11453
reference study
Binary_log_event
^
|
|
Query_event Log_event
\ /
<<virtual>>\ /
\ /
Query_log_event
define query_event status_vars
libbinlogevents/include/statement_events.h > Query_event /** @addtogroup Replication @{
@file statement_events.h
@brief Contains the classes representing statement events occurring in the
replication stream. Each event is represented as a byte sequence with logical
divisions as event header, event specific data and event footer. The header
and footer are common to all the events and are represented as two different
subclasses.
*/ [definition of Query_event_status_vars](https://github.com/mysql/mysql-server/blob/beb865a960b9a8a16cf999c323e46c5b0c67f21f/libbinlogevents/include/statement_events.h#L463-L532) [comment on status_vars](https://github.com/mysql/mysql-server/blob/beb865a960b9a8a16cf999c323e46c5b0c67f21f/libbinlogevents/include/statement_events.h#L153-L446)
construct query_event
libbinlogevents/include/statement_events.cpp > Query_event constructor of Query_event
event_reader
libbinlogevents/include/event_reader.h /** @addtogroup Replication @{
@file event_reader.h
@brief Contains the class responsible for deserializing fields of an event previously stored in a buffer. */
write Query_log_event
sql/log_event.h /** @file sql/log_event.h
@brief Binary log event definitions. This includes generic code
common to all types of log events, as well as specific code for each
type of log event.
@addtogroup Replication
@{
*/
/*
One class for each type of event.
Two constructors for each class:
- one to create the event for logging (when the server acts as a master),
called after an update to the database is done,
which accepts parameters like the query, the database, the options for LOAD
DATA INFILE...
- one to create the event from a packet (when the server acts as a slave),
called before reproducing the update, which accepts parameters (like a
buffer). Used to read from the master, from the relay log, and in
mysqlbinlog. This constructor must be format-tolerant.
*/
/**
Query_log_event::write().
@note
In this event we have to modify the header to have the correct
EVENT_LEN_OFFSET as we don't yet know how many status variables we
will print!
*/
reference to parse query_event.status_vars
MySQL Documentaition mysql-server/client mysql-server/statement-events -> latest
information on columns
TableMapEvent
[…, ‘column_count’, ‘column_schemas’, ‘columns’, …]
column_schemas: List[Dict]
COLUMN_NAME COLLATION_NAME CHARACTER_SET_NAME COLUMN_COMMENT COLUMN_TYPE COLUMN_KEY ORDINAL_POSITION
columns: List[Column]
[…, ‘character_set_name’, ‘collation_name’, ‘comment’, ‘data’, ‘is_primary’, ‘name’, ‘serializable_data’, ‘type’, ‘type_is_bool’, ‘unsigned’, ‘zerofill’], …]
WriteRowsEvent
[…, ‘columns’, ‘columns_present_bitmap’, ‘number_of_columns’, …]
##