MySQL 과부하 성능 문제

기본적으로 SQL 쿼리에 300 초 이상 걸리는 큰 제작 문제가 있습니다.MySQL 과부하 성능 문제

우리는 SQL 서버로 MySQL을 사용하고 있으며 우리 테이블은 MyISAM을 사용하고 있습니다.

문제가되는 쿼리는 두 테이블 간의 기본 쿼리이며 정렬 및 그룹 기준입니다. 문제는 각 테이블에 3 천만 개가 넘는 행이 있다는 것입니다. ids_stat 테이블에

SELECT  
    i.id_stat as idStat,  
    SUM(srv0.duree_st) AS dureeAppelTotale,  
    _date as dateI,  
    DATE_FORMAT(srv0._date,'%d/%m') AS datekey,  
    DATE_FORMAT(_date,'%d%m%Y') AS datekeyGroup,  
    heure,  
    SUM(srv0.nombre_st) AS nbAppel 
FROM ids_stat i , SRV201606 srv0 
WHERE i.id_rubrique = srv0.id_rubrique  
    AND i.id_appli = 24071  
    AND srv0._date >= '2016-06-01 00:00:00'  
    AND srv0._date <= '2016-07-01 00:00:00' 
GROUP BY id_stat, datekeyGroup, heure 
ORDER BY _date

인덱스는 다음과 같습니다 :

SRV0 테이블에

show index from ids_stat; 
+----------+------------+---------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+ 
| Table | Non_unique | Key_name   | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | 
+----------+------------+---------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+ 
| ids_stat |   0 | PRIMARY    |   1 | id_rubrique | A   |  7435537 |  NULL | NULL |  | BTREE  |   | 
| ids_stat |   1 | id_appli_index  |   1 | id_appli  | A   |  13593 |  NULL | NULL |  | BTREE  |   | 
| ids_stat |   1 | date_creation_index |   1 | date_creation | A   |  2478512 |  NULL | NULL |  | BTREE  |   |

, 인덱스는 다음과 같습니다 또한

show index from SRV201606; 
+-----------+------------+----------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+ 
| Table  | Non_unique | Key_name  | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | 
+-----------+------------+----------------+--------------+----------------+-----------+-------------+----------+--------+------+------------+---------+ 
| SRV201606 |   0 | PRIMARY  |   1 | _date   | A   |  NULL |  NULL | NULL |  | BTREE  |   | 
| SRV201606 |   0 | PRIMARY  |   2 | heure   | A   |  NULL |  NULL | NULL |  | BTREE  |   | 
| SRV201606 |   0 | PRIMARY  |   3 | minute   | A   |  NULL |  NULL | NULL |  | BTREE  |   | 
| SRV201606 |   0 | PRIMARY  |   4 | id_plaque  | A   |  NULL |  NULL | NULL |  | BTREE  |   | 
| SRV201606 |   0 | PRIMARY  |   5 | id_rubrique | A   |  NULL |  NULL | NULL |  | BTREE  |   | 
| SRV201606 |   0 | PRIMARY  |   6 | id_combinaison | A   | 27472765 |  NULL | NULL |  | BTREE  |   | 
| SRV201606 |   1 | id_rubrique |   1 | id_rubrique | A   |  NULL |  NULL | NULL |  | BTREE  |   | 
| SRV201606 |   1 | id_combinaison |   1 | id_combinaison | A   |  NULL |  NULL | NULL |  | BTREE  |   |

여기

이 쿼리의 가짜 예입니다 다음은 쿼리에 대한 설명입니다.

+----+-------------+-------+------+------------------------+----------------+---------+-----------------------------+------+---------------------------------+ 
| id | select_type | table | type | possible_keys   | key   | key_len | ref       | rows | Extra       | 
+----+-------------+-------+------+------------------------+----------------+---------+-----------------------------+------+---------------------------------+ 
| 1 | SIMPLE  | i  | ref | PRIMARY,id_appli_index | id_appli_index | 2  | const      | 153 | Using temporary; Using filesort | 
| 1 | SIMPLE  | srv0 | ref | PRIMARY,id_rubrique | id_rubrique | 8  | StatVoxProdV2.i.id_rubrique | 16 | Using where      |

SQL 엔진에 의해 내부 조인으로 변환되므로 무해한 곳에서 만든 이러한 이전 조인도 읽습니다.

MySQL 서버를 향상시킬 수있는 방법을 모색하는 데 도움을 줄 수 있습니까? 우리를 도울 수있는 구성이 있습니까?

많은 감사 의견에 질문으로

[편집] 가 여기에 그는 버전 :

SELECT VERSION();

+------------+ 
| VERSION() | 
+------------+ 
| 5.0.45-log |

서버 변수 :

show Variables; 
+---------------------------------+----------------------------------+ 
| Variable_name     | Value       | 
+---------------------------------+----------------------------------+ 
| auto_increment_increment  | 1        | 
| auto_increment_offset   | 1        | 
| automatic_sp_privileges   | ON        | 
| back_log      | 50        | 
| basedir       | /usr/local/      | 
| binlog_cache_size    | 32768       | 
| bulk_insert_buffer_size   | 8388608       | 
| character_set_client   | utf8        | 
| character_set_connection  | utf8        | 
| character_set_database   | latin1       | 
| character_set_filesystem  | binary       | 
| character_set_results   | utf8        | 
| character_set_server   | latin1       | 
| character_set_system   | utf8        | 
| character_sets_dir    | /usr/local/share/mysql/charsets/ | 
| collation_connection   | utf8_general_ci     | 
| collation_database    | latin1_swedish_ci    | 
| collation_server    | latin1_swedish_ci    | 
| completion_type     | 0        | 
| concurrent_insert    | 1        | 
| connect_timeout     | 5        | 
| datadir       | THEDATADIR   | 
| date_format      | %Y-%m-%d       | 
| datetime_format     | %Y-%m-%d %H:%i:%s    | 
| default_week_format    | 0        | 
| delay_key_write     | ON        | 
| delayed_insert_limit   | 100        | 
| delayed_insert_timeout   | 300        | 
| delayed_queue_size    | 1000        | 
| div_precision_increment   | 4        | 
| engine_condition_pushdown  | OFF        | 
| expire_logs_days    | 32        | 
| flush       | OFF        | 
| flush_time      | 0        | 
| ft_boolean_syntax    | + -><()~*:""&|     | 
| ft_max_word_len     | 84        | 
| ft_min_word_len     | 4        | 
| ft_query_expansion_limit  | 20        | 
| ft_stopword_file    | (built-in)      | 
| group_concat_max_len   | 1024        | 
| have_archive     | NO        | 
| have_bdb      | NO        | 
| have_blackhole_engine   | NO        | 
| have_compress     | YES        | 
| have_crypt      | YES        | 
| have_csv      | NO        | 
| have_dynamic_loading   | YES        | 
| have_example_engine    | NO        | 
| have_federated_engine   | NO        | 
| have_geometry     | YES        | 
| have_innodb      | DISABLED       | 
| have_isam      | NO        | 
| have_merge_engine    | YES        | 
| have_ndbcluster     | NO        | 
| have_openssl     | NO        | 
| have_ssl      | NO        | 
| have_query_cache    | YES        | 
| have_raid      | NO        | 
| have_rtree_keys     | YES        | 
| have_symlink     | YES        | 
| hostname      | THEHOSTNAME     | 
| init_connect     |         | 
| init_file      |         | 
| init_slave      |         | 
| innodb_additional_mem_pool_size | 1048576       | 
| innodb_autoextend_increment  | 8        | 
| innodb_buffer_pool_awe_mem_mb | 0        | 
| innodb_buffer_pool_size   | 1048576       | 
| innodb_checksums    | ON        | 
| innodb_commit_concurrency  | 0        | 
| innodb_concurrency_tickets  | 500        | 
| innodb_data_file_path   |         | 
| innodb_data_home_dir   |         | 
| innodb_doublewrite    | ON        | 
| innodb_fast_shutdown   | 1        | 
| innodb_file_io_threads   | 4        | 
| innodb_file_per_table   | OFF        | 
| innodb_flush_log_at_trx_commit | 1        | 
| innodb_flush_method    |         | 
| innodb_force_recovery   | 0        | 
| innodb_lock_wait_timeout  | 50        | 
| innodb_locks_unsafe_for_binlog | OFF        | 
| innodb_log_arch_dir    |         | 
| innodb_log_archive    | OFF        | 
| innodb_log_buffer_size   | 1048576       | 
| innodb_log_file_size   | 5242880       | 
| innodb_log_files_in_group  | 2        | 
| innodb_log_group_home_dir  |         | 
| innodb_max_dirty_pages_pct  | 90        | 
| innodb_max_purge_lag   | 0        | 
| innodb_mirrored_log_groups  | 1        | 
| innodb_open_files    | 300        | 
| innodb_rollback_on_timeout  | OFF        | 
| innodb_support_xa    | ON        | 
| innodb_sync_spin_loops   | 20        | 
| innodb_table_locks    | ON        | 
| innodb_thread_concurrency  | 8        | 
| innodb_thread_sleep_delay  | 10000       | 
| interactive_timeout    | 28800       | 
| join_buffer_size    | 131072       | 
| key_buffer_size     | 402653184      | 
| key_cache_age_threshold   | 300        | 
| key_cache_block_size   | 1024        | 
| key_cache_division_limit  | 100        | 
| language      | /usr/local/share/mysql/english/ | 
| large_files_support    | ON        | 
| large_page_size     | 0        | 
| large_pages      | OFF        | 
| lc_time_names     | en_US       | 
| license       | GPL        | 
| local_infile     | ON        | 
| locked_in_memory    | OFF        | 
| log        | OFF        | 
| log_bin       | ON        | 
| log_bin_trust_function_creators | OFF        | 
| log_error      |         | 
| log_queries_not_using_indexes | OFF        | 
| log_slave_updates    | OFF        | 
| log_slow_queries    | ON        | 
| log_warnings     | 1        | 
| long_query_time     | 10        | 
| low_priority_updates   | OFF        | 
| lower_case_file_system   | OFF        | 
| lower_case_table_names   | 0        | 
| max_allowed_packet    | 1047552       | 
| max_binlog_cache_size   | 4294967295      | 
| max_binlog_size     | 104857600      | 
| max_connect_errors    | 10        | 
| max_connections     | 100        | 
| max_delayed_threads    | 20        | 
| max_error_count     | 64        | 
| max_heap_table_size    | 134217728      | 
| max_insert_delayed_threads  | 20        | 
| max_join_size     | 4294967295      | 
| max_length_for_sort_data  | 1024        | 
| max_prepared_stmt_count   | 16382       | 
| max_relay_log_size    | 0        | 
| max_seeks_for_key    | 4294967295      | 
| max_sort_length     | 1024        | 
| max_sp_recursion_depth   | 0        | 
| max_tmp_tables     | 32        | 
| max_user_connections   | 0        | 
| max_write_lock_count   | 4294967295      | 
| multi_range_count    | 256        | 
| myisam_data_pointer_size  | 6        | 
| myisam_max_sort_file_size  | 2147483647      | 
| myisam_recover_options   | OFF        | 
| myisam_repair_threads   | 1        | 
| myisam_sort_buffer_size   | 67108864       | 
| myisam_stats_method    | nulls_unequal     | 
| net_buffer_length    | 16384       | 
| net_read_timeout    | 30        | 
| net_retry_count     | 10        | 
| net_write_timeout    | 60        | 
| new        | OFF        | 
| old_passwords     | OFF        | 
| open_files_limit    | 4096        | 
| optimizer_prune_level   | 1        | 
| optimizer_search_depth   | 62        | 
| pid_file      | /usr2/mysql/var/run/mysqld.pid | 
| port       | 3306        | 
| preload_buffer_size    | 32768       | 
| profiling      | OFF        | 
| profiling_history_size   | 15        | 
| protocol_version    | 10        | 
| query_alloc_block_size   | 8192        | 
| query_cache_limit    | 1048576       | 
| query_cache_min_res_unit  | 4096        | 
| query_cache_size    | 33554432       | 
| query_cache_type    | ON        | 
| query_cache_wlock_invalidate | OFF        | 
| query_prealloc_size    | 8192        | 
| range_alloc_block_size   | 2048        | 
| read_buffer_size    | 2093056       | 
| read_only      | OFF        | 
| read_rnd_buffer_size   | 8384512       | 
| relay_log_purge     | ON        | 
| relay_log_space_limit   | 0        | 
| rpl_recovery_rank    | 0        | 
| secure_auth      | OFF        | 
| secure_file_priv    |         | 
| server_id      | 2        | 
| skip_external_locking   | ON        | 
| skip_networking     | OFF        | 
| skip_show_database    | OFF        | 
| slave_compressed_protocol  | OFF        | 
| slave_load_tmpdir    | /usr2/mysql/tmp/     | 
| slave_net_timeout    | 3600        | 
| slave_skip_errors    | ALL        | 
| slave_transaction_retries  | 10        | 
| slow_launch_time    | 2        | 
| socket       | /usr2/mysql/var/run/mysqld.sock | 
| sort_buffer_size    | 2097144       | 
| sql_big_selects     | ON        | 
| sql_mode      |         | 
| sql_notes      | ON        | 
| sql_warnings     | OFF        | 
| ssl_ca       |         | 
| ssl_capath      |         | 
| ssl_cert      |         | 
| ssl_cipher      |         | 
| ssl_key       |         | 
| storage_engine     | MyISAM       | 
| sync_binlog      | 0        | 
| sync_frm      | ON        | 
| system_time_zone    | CET        | 
| table_cache      | 512        | 
| table_lock_wait_timeout   | 50        | 
| table_type      | MyISAM       | 
| thread_cache_size    | 8        | 
| thread_stack     | 196608       | 
| time_format      | %H:%i:%s       | 
| time_zone      | SYSTEM       | 
| timed_mutexes     | OFF        | 
| tmp_table_size     | 134217728      | 
| tmpdir       | /usr2/mysql/tmp/     | 
| transaction_alloc_block_size | 8192        | 
| transaction_prealloc_size  | 4096        | 
| tx_isolation     | REPEATABLE-READ     | 
| updatable_views_with_limit  | YES        | 
| version       | 5.0.45-log      | 
| version_comment     | Source distribution    | 
| version_compile_machine   | i686        | 
| version_compile_os    | pc-linux-gnu      | 
| wait_timeout     | 28800       |

ids_stat 스키마 :

SHOW CREATE TABLE ids_stat; 
CREATE TABLE `ids_stat` (
    `id_rubrique` bigint(20) NOT NULL default '0', 
    `id_stat` int(10) default NULL, 
    `id_appli` smallint(5) unsigned NOT NULL default '0', 
    `nom_rubrique` varchar(240) NOT NULL, 
    `date_creation` timestamp NOT NULL default CURRENT_TIMESTAMP, 
    PRIMARY KEY (`id_rubrique`), 
    KEY `id_appli_index` (`id_appli`), 
    KEY `date_creation_index` (`date_creation`) 
) ENGINE=MyISAM DEFAULT CHARSET=latin1

그리고 SRV0 예 스키마 :

CREATE TABLE `SRV201606` (
    `id_rubrique` bigint(20) NOT NULL default '0', 
    `id_plaque` tinyint(3) unsigned NOT NULL default '0', 
    `id_combinaison` smallint(5) unsigned NOT NULL default '0', 
    `_date` date NOT NULL default '0000-00-00', 
    `heure` tinyint(3) NOT NULL default '0', 
    `minute` tinyint(3) NOT NULL default '0', 
    `nombre_entree` mediumint(8) unsigned default '0', 
    `nombre_st` mediumint(8) unsigned default NULL, 
    `duree_st` mediumint(8) unsigned default NULL, 
    `flag` tinyint(3) unsigned NOT NULL default '0', 
    PRIMARY KEY (`_date`,`heure`,`minute`,`id_plaque`,`id_rubrique`,`id_combinaison`), 
    KEY `id_rubrique` (`id_rubrique`), 
    KEY `id_combinaison` (`id_combinaison`) 
) ENGINE=MyISAM DEFAULT CHARSET=latin1

출처

2016-06-22 gfyhser

먼저, 조인을 수정해야합니다. 오래된 구식 스타일입니다. 자세한 내용은 여기를 참조하십시오. http://sqlblog.com/blogs/aaron_bertrand/archive/2009/10/08/bad-habits-to-kick-using-old-style-joins.aspx –

테이블의 색인에 대한 정보를 질문. –

먼저, @RichBenner 슬프지만,'INNER JOIN'을 다음과 같이 사용하십시오 :'WHERE i.id_rubrique = srv0.id_rubrique'. 두 번째로, 테이블의 색인을 생성해야합니다. '_date','id_stat, datekeyGroup, heure'및'SELECT' 다음의 모든 필드에 색인을 추가하십시오. 'EXPLAIN'을 추가 할 수 있습니다. select, see, what happen. – vaso123

이 내 완성 된 대답!

아니다

큰 테이블에 대해서만이 쿼리를 실행할 수 있고 실행 시간과 설명을 제공 할 수 있습니까?

변경 테이블 SRV201606 추가 키 idx_rubrique_date (_date, id_rubrique);

SELECT 
    i.*,  
    SUM(srv0.duree_st) AS dureeAppelTotale,  
    srv0._date AS dateI,  
    DATE_FORMAT(srv0._date,'%d/%m') AS datekey,  
    DATE_FORMAT(_date,'%d%m%Y') AS datekeyGroup,  
    heure,  
    SUM(srv0.nombre_st) AS nbAppel 
FROM (
    SELECT  
     id_rubrique 
     , id_stat 
    FROM ids_stat 
    WHERE  
    id_appli = 24071  
    GROUP BY id_stat 
) AS i 
LEFT JOIN SRV201606 srv0 ON i.id_rubrique = srv0.id_rubrique 
WHERE 
    srv0._date >= '2016-06-01'  
AND 
    srv0._date <= '2016-07-01' 
GROUP BY id_stat, _date, heure 
ORDER BY _date;

출처

2016-06-22 11:43:51

이 쿼리는 실행하는 데 17.29 초가 걸립니다. 여기에 설명이 있습니다. http : //pastebin.com/qHDwsUmN 또한이 쿼리는 원래 쿼리보다 적은 결과를 제공합니다. – gfyhser

결과가 좋습니까? –

내 첫 코멘트를 업데이트했습니다 :-) – gfyhser

당신은 너무

CREATE TABLE `SRV201606` (
... 

KEY `id_rubrique` (`id_rubrique`),

KEY `id_rubrique` (`_date`,`id_rubrique`),

또는 대안

KEY `date-id_rubrique` (`_date`,`id_rubrique`),

출처

2016-06-22 12:04:04 Jaydee

당신이 EXPLAIN 날짜에서 시간을 제거 시도하고 게시 할 수에 _date를 추가하는 시도 할 수 , 다음과 같이 : -

SELECT  
    i.id_stat as idStat,  
    SUM(srv0.duree_st) AS dureeAppelTotale,  
    _date as dateI,  
    DATE_FORMAT(srv0._date,'%d/%m') AS datekey,  
    DATE_FORMAT(_date,'%d%m%Y') AS datekeyGroup,  
    heure,  
    SUM(srv0.nombre_st) AS nbAppel 
FROM ids_stat i 
INNER JOIN SRV201606 srv0 
ON i.id_rubrique = srv0.id_rubrique  
WHERE i.id_appli = 24071  
AND srv0._date BETWEEN '2016-06-01' AND '2016-07-01' 
GROUP BY id_stat, 
      datekeyGroup, 
      heure 
ORDER BY _date

MySQL은 반환 된 각 행에 대해 날짜/시간으로 날짜 변환을 시도하고 있다고 생각합니다.

출처

2016-06-23 16:12:02 Kickstart

WHERE i.id_rubrique = srv0.id_rubrique  
    AND i.id_appli = 24071  
    AND srv0._date >= '2016-06-01 00:00:00'  
    AND srv0._date <= '2016-07-01 00:00:00'

둘 다 표시 하시겠습니까? 제안 대신이 일을 :

AND srv0._date >= '2016-06-01'  
    AND srv0._date < '2016-06-01' + INTERVAL 1 MONTH

을 i로 시작 촉진하기 위해이 인덱스를 필요 :

i: (id_appli, id_rubrique, id_stat) -- in this order; and is 'covering' 
srv0: (id_rubrique, _date)   -- '=' part first, then 'range'

을 srv0로 시작 촉진하기 위해이 인덱스를 필요 :

i: (id_rubrique, id_appli, id_stat) -- in this order; and is 'covering' 
srv0: (_date)      -- adding anything after `_date` is useless

을 줄 것이다 모든 인덱스를 추가 최적화 프로그램은 테이블로 시작하여 효율적으로 수행 할 수있는 옵션을 제공합니다.

출처

2016-06-28 21:56:27

MySQL 과부하 성능 문제

답변

관련 문제