I'm working with Mediawiki 1.25.1 on CentOS 7.2 running Apache 2.4.3. I'm working with Mediawiki 1.26.4 on CentOS 7.2 running Apache 2.4.6. I am experiencing an Internal Server Error after editing a page and submitting it:There is no additional information in /var/log/httpd/error_log. According to Manual:How to debug, I added the following to LocalSettings, and then restarted Apache:$ sudo tail -7 LocalSettings.php# Added by JW for debugging (JUN 2016). Keep commented unless needed.$wgShowExceptionDetails=true;$wgShowSQLErrors=true;error_reporting( -1 );ini_set( 'display_errors', 1 );?>Adding $wgShowExceptionDetails, error_reporting and friends does not provide any additional information that we can find. error_log is clean, access_log is clean. We enabled error_log = php_errors.log, but the server does not produce a php_errors.log. We have no idea where else to look.We ran mysqlcheck my_wiki --auto-repair --user=... --password=... and it reports no errors. apachectl configtest immediately returns Syntax OK. However, the problem persists.We also performed an upgrade to MediaWiki 1.26.4 (from 1.25.1) in hopes of fixing this internal server error with no joy. It still persists, and I still cannot get additional information about it.QUESTIONS: How does one really enable debugging information? How can we gather more information about the problem plaguing this server?(Please provide actionable items in response to the question; and please don't provide off-site links telling us to try some of the stuff on some other page that may or may not work. And please provide answers for the specific question that was asked, and not other questions that were not asked.)Here's the most recent useless error message from error_log after upgrading to MediaWiki 1.26. Its from the tail of /var/log/httpd/error_log when a request was submitted to update a wiki page....[Fri Oct 28 21:05:04.456126 2016] [suexec:notice] [pid 1053] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)[Fri Oct 28 21:05:04.457260 2016] [:notice] [pid 1053] ModSecurity for Apache/2.7.3 (http://www.modsecurity.org/) configured.[Fri Oct 28 21:05:04.457283 2016] [:notice] [pid 1053] ModSecurity: APR compiled version=1.4.8; loaded version=1.4.8[Fri Oct 28 21:05:04.457290 2016] [:notice] [pid 1053] ModSecurity: PCRE compiled version=8.32 ; loaded version=8.32 2012-11-30[Fri Oct 28 21:05:04.457295 2016] [:notice] [pid 1053] ModSecurity: LUA compiled version=Lua 5.1[Fri Oct 28 21:05:04.457299 2016] [:notice] [pid 1053] ModSecurity: LIBXML compiled version=2.9.1[Fri Oct 28 21:05:04.495066 2016] [auth_digest:notice] [pid 1053] AH01757: generating secret for digest authentication ...[Fri Oct 28 21:05:04.632965 2016] [mpm_prefork:notice] [pid 1053] AH00163: Apache/2.4.6 (CentOS) OpenSSL/1.0.1e-fips configured -- resuming normal operations[Fri Oct 28 21:05:04.632994 2016] [core:notice] [pid 1053] AH00094: Command line: '/usr/sbin/httpd -D FOREGROUND'A couple of folks have commented about the startup messages. I did not realize it was abnormal since Apache always did it. We recently made the following change to address it (but it did not help out no information problem):# diff /etc/httpd/conf.d/mpm_prefork.conf.bu /etc/httpd/conf.d/mpm_prefork.conf9,13c9,13< StartServers       1< MinSpareServers    1< MaxSpareServers    5< ServerLimit       10< MaxClients        10---> StartServers       4> MinSpareServers    4> MaxSpareServers    8> ServerLimit       32> MaxClients        32None of us are professional admins, so we don't know if its enough to resolve the issue. We are a bunch of free and open software developers who run a web server and wiki to help users.Here is the maximum brevity /etc/php.ini:# cat /etc/php.ini | egrep -v '(^;|^\[)' | sed '/^$/d'engine = Onshort_open_tag = Offasp_tags = Offprecision = 14output_buffering = 4096zlib.output_compression = Offimplicit_flush = Offunserialize_callback_func =serialize_precision = 17disable_functions =disable_classes =zend.enable_gc = Onexpose_php = Onmax_execution_time = 30max_input_time = 60memory_limit = 128Merror_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICTdisplay_errors = Offdisplay_startup_errors = Offlog_errors = Onlog_errors_max_len = 1024ignore_repeated_errors = Offignore_repeated_source = Offreport_memleaks = Ontrack_errors = Offhtml_errors = Onvariables_order = GPCSrequest_order = GPregister_argc_argv = Offauto_globals_jit = Onpost_max_size = 8Mauto_prepend_file =auto_append_file =default_mimetype = text/htmldoc_root =user_dir =enable_dl = Offfile_uploads = Onupload_max_filesize = 2Mmax_file_uploads = 20allow_url_fopen = Onallow_url_include = Offdefault_socket_timeout = 60cli_server.color = Ondate.timezone = UTCpdo_mysql.cache_size = 2000pdo_mysql.default_socket=SMTP = localhostsmtp_port = 25sendmail_path = /usr/sbin/sendmail -t -imail.add_x_header = Onsql.safe_mode = Offodbc.allow_persistent = Onodbc.check_persistent = Onodbc.max_persistent = -1odbc.max_links = -1odbc.defaultlrl = 4096odbc.defaultbinmode = 1ibase.allow_persistent = 1ibase.max_persistent = -1ibase.max_links = -1ibase.timestampformat = %Y-%m-%d %H:%M:%Sibase.dateformat = %Y-%m-%dibase.timeformat = %H:%M:%Smysql.allow_local_infile = Onmysql.allow_persistent = Onmysql.cache_size = 2000mysql.max_persistent = -1mysql.max_links = -1mysql.default_port =mysql.default_socket =mysql.default_host =mysql.default_user =mysql.default_password =mysql.connect_timeout = 60mysql.trace_mode = Offmysqli.max_persistent = -1mysqli.allow_persistent = Onmysqli.max_links = -1mysqli.cache_size = 2000mysqli.default_port = 3306mysqli.default_socket =mysqli.default_host =mysqli.default_user =mysqli.default_pw =mysqli.reconnect = Offmysqlnd.collect_statistics = Onmysqlnd.collect_memory_statistics = Offpgsql.allow_persistent = Onpgsql.auto_reset_persistent = Offpgsql.max_persistent = -1pgsql.max_links = -1pgsql.ignore_notice = 0pgsql.log_notice = 0sybct.allow_persistent = Onsybct.max_persistent = -1sybct.max_links = -1sybct.min_server_severity = 10sybct.min_client_severity = 10bcmath.scale = 0session.save_handler = filessession.use_cookies = 1session.use_only_cookies = 1session.name = PHPSESSIDsession.auto_start = 0session.cookie_lifetime = 0session.cookie_path = /session.cookie_domain =session.cookie_httponly =session.serialize_handler = phpsession.gc_probability = 1session.gc_divisor = 1000session.gc_maxlifetime = 1440session.bug_compat_42 = Offsession.bug_compat_warn = Offsession.referer_check =session.cache_limiter = nocachesession.cache_expire = 180session.use_trans_sid = 0session.hash_function = 0session.hash_bits_per_character = 5url_rewriter.tags = a=href,area=href,frame=src,input=src,form=fakeentrymssql.allow_persistent = Onmssql.max_persistent = -1mssql.max_links = -1mssql.min_error_severity = 10mssql.min_message_severity = 10mssql.compatability_mode = Offmssql.secure_connection = Offtidy.clean_output = Offsoap.wsdl_cache_enabled=1soap.wsdl_cache_dir=/tmpsoap.wsdl_cache_ttl=86400soap.wsdl_cache_limit = 5ldap.max_links = -1
Internal Server Error; enable debugging provides no additional information
php;apache;mediawiki
I think we figured out how to diagnose the issue. All this time we were focusing on error logs because we consider a 500 server crash an error, and we specified errors should be logged to error_log in the configuration files.Out of desperation, we grep'ed the entire file system for *Internal Server Error. Lo and behold, mod_security is not honoring the configuration settings:# grep -IR 'Internal Server Error' /var/log.../var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error/var/log/httpd/modsec_audit.log:HTTP/1.1 500 Internal Server Error.../var/log/httpd/modsec_audit.log-20161024:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161024:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161024:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161024:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161024:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161024:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161024:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161024:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161024:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161016:HTTP/1.0 500 Internal Server Error/var/log/httpd/modsec_audit.log-20161016:HTTP/1.0 500 Internal Server Error     ...<no error_log entries>I'm OK with mod_security logging to their own file. I take exception that they did not log to where we told them to through php.ini, and it has wasted months of man hours and trouble shooting. All they had to do was write an occasional message into error_log and this would have been fixed months ago.