store | blogs | forums | twitter | facebook | wiki | downloads | support portal
Atomic Secure Linux
It is currently Sat Oct 19, 2019 7:20 pm

» Feed - Atomicorp

All times are UTC - 5 hours [ DST ]




Post new topic Reply to topic  [ 2 posts ] 
Author Message
 Post subject: Mariadb eating disk - help!
Unread postPosted: Mon Feb 22, 2016 1:15 pm 
Offline
Forum Regular
Forum Regular

Joined: Mon Sep 16, 2013 10:10 am
Posts: 112
Location: Lahore
I've reworked the my setup to eliminate nginx and work solely with apache. This time I decided to use MariaDB 10.1 (innodb db), boy was this a mistake.

The application is a news website, apart from the standard news insertions to the db, news article read analytics are also recorded to the db. That means one sql entry for each page view (30-50 visits/second). Before all this the db was set to myisam, we were experiencing issues with table crashes so we decided to switch to innodb.

The problem: Since the move, mariadb has started to eat up all the space in /tmp. I don't have /tmp partitioned to a separate partition so it ends up eating all the disk. I've tried looking for something regarding this online but haven't been able to find anything that would help.

CPU consumption stands around 40-50% and overall server ram usage stands firmly at 5GB, hasn't gone above that the whole day today.

Server specs:
Xeon(R) CPU E5-1650 v3
64GB DDR4 ECC RAM
2 x 240GB SSDs (Software Raid 1)
CentOS 7.2

server.conf
Code:
[server]

[mysqld]

skip_name_resolve
default_storage_engine = InnoDB
tmp_table_size = 512M
max_heap_table_size = 512M
max_connect_errors = 10000
innodb_flush_method = O_DIRECT
innodb_log_files_in_group = 2
innodb_log_file_size = 512M
innodb_flush_log_at_trx_commit = 1
innodb_buffer_pool_size = 15G
innodb_buffer_pool_instances = 15
innodb_log_buffer_size = 128M
innodb_write_io_threads = 8
innodb_read_io_threads = 8

# LOGGING #

log_error = /var/log/mysql/mysqld.log
slow_query_log = 0
slow_query_log_file = /var/log/mysql/mysql-slow.log
long_query_time = 1
#log_queries_not_using_indexes = 1

[galera]

[embedded]

[mariadb]

[mariadb-10.1]

[mysqld_safe]


Here's a screenshot from newrelic for the past 3 hours

Image


What am I doing wrong? Please help!


Top
 Profile  
Reply with quote  
 Post subject: Re: Mariadb eating disk - help!
Unread postPosted: Tue Feb 23, 2016 9:33 am 
Offline
Forum Regular
Forum Regular

Joined: Tue Aug 01, 2006 2:45 pm
Posts: 573
Location: Netherlands
Assuming the metrics provided are correct, it seems that the disk was incredibly slow between 9:10 and 9:20. This caused processes waiting on the disk, temporarily slowing down everything. Perhaps a RAID rebuild? Hardware issues? Overbooked public cloud?

_________________
Lemonbit Internet Dedicated Server Management


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 2 posts ] 

» Feed - Atomicorp

All times are UTC - 5 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group