Support

Home Forums Event Espresso Premium Irregular 404 errors – continued

Irregular 404 errors – continued

Posted: December 5, 2019 at 5:43 am

Viewing 15 reply threads


Joost

December 5, 2019 at 5:43 am

A few months ago I reported this nasty issue (now closed for replies, so I create a new entry)

https://eventespresso.com/topic/irregular-404-errors/

We have done lots of experiments and testing (quite tedious given the seemingly random occurrence of the 404 errors). Finally, we seem to close in to it. We found a correlation between the 404 errors and two plugins: “Page links to” and “Redirection”. Both plugins They have in common that they redirect urls. They are used for other post types and not actively for EE events, but seem to have an effect on EE pages as well. There is also a time effect. If we reset the server, the issue seem to dissappear but over time (hours, days) it reappears.

Note that the issue appears for EE pages only, not for other post types and that it appears with at least two plugins, so it cannot be attributed to a single malfunctioning plugin. The common factor still seems to be EE. Apparently the EE urls/permalinks are handled differently than others.

I hope this gives some ideas where to look for a solution (other than disabling the plugins)


Tony

  • Support Staff

December 5, 2019 at 6:23 am

Hi there,

We recently did some digging into this issue and made a change to try and battle harden EE’s rewrite rules a little more on a separate branch which is currently being tested.

If you’d like to test that version it is is available HERE, it is very similar to the release version of EE.


Joost

December 6, 2019 at 2:03 pm

Thanks Tony, I’m very glad you’re looking in to this! Currently, we’re approaching a major event, need a stable site and cannot risk testing. We’ve sacrificed the two plugins for now and keep our fingers crossed. But from mid January we have more room for testing. I’m looking forward for a solution.


Tony

  • Support Staff

December 9, 2019 at 3:54 am

Currently, we’re approaching a major event, need a stable site and cannot risk testing.

Totally understandable 🙂

The change we’ve made basically changes when the code to register our re-write rules is run, everything else remains the same so hopefully, it should solve the problem. With the number of possible causes, we can’t say for certain until we push it live and get some feedback from user experiencing the issue so we’ll be testing to ensure the changes don’t affect any of EE’s functionality as is.

We’ll likely push the change in the next update if no issues show up in testing.


Joost

December 16, 2019 at 1:07 pm

Hi Tony,

Unfortunately, our workaround to disable the two plugins appeared to be insufficient. Therefor, we installed your test version. Again unfortunately, that does not solve the problem either. Even worse, the problem gets bigger and bigger over time. Where in the past rebooting the server, resetting PHP, disabling and enabling EE gave some temporary relief, this doesn’t help anymore. It seems that we approaching some kind of EE limit. Now we do have a fairly large database of events (278). Could that contribute to the instability?

Joost


Joost

December 17, 2019 at 12:34 am

An emergency update: by now nothing helps anymore: de- and re-activating EE, rebooting the server, resetting PHP. This all helped temporarily, but the time between the re-occurence of the problem became shorter and shorter. De- and re-activating EE seemed the last resort with most effect. But that is failing now as well. The event part of our site has totally crashed now. The rest of the site is still working.

So, with a major event coming up, we are in big trouble!

We will try to clean the EE database, hoping that that has an effect. But what is the best way to do that? I guess we have to delete events with all dependent data. Just removing an event will not make sense if the transactions and participants remain. That is much more data than the event itself.


Joost

December 17, 2019 at 12:59 am

To add to my previous emergency update: we have made several attempts to clean the database. We can delete events of course but related registrations and transactions stay in place. If we try to delete those seperately, EE crashes. So we are now lost what to do….


Tony

  • Support Staff

December 17, 2019 at 3:11 am

Hi Joost,

I can’t think of any ‘EE limit’ that you would be hitting here and ~270 events is not really not a ‘high number’ of events so that shouldn’t be causing any issues.

Do you have any evidence to support that the issue is the number of events/registrations?

The only reason I ask is you could spend a large amount of time working through ‘cleaning’ the database and still be in the exact same situation.

I run multiple test sites on different hosting types in an effort to test EE on the types of hosting user have, so I have sites using really cheap shared hosting, mid-range shared and VPS packages which never show any issues with permalinks. I don’t know your registration numbers but you may have higher numbers of registrations than I, however, those don’t load everywhere so it should not cause issues with permalinks.

The event part of our site has totally crashed now.

When you say it’s crashed, what is happening? Do all events show 404? A white screen? Another error?

We will try to clean the EE database, hoping that that has an effect. But what is the best way to do that?

Before you started altering the database, did you create a backup? Manually ‘cleaning’ the database is not a good idea here, there are multiple relationships within EE’s data and if you miss any you end up with orphaned data which if/when it loads in the EE ui, will cause it to crash.

To add to my previous emergency update: we have made several attempts to clean the database.

Can you add some details of what it is you are doing when you attempt to clean the database, please?

We can delete events of course but related registrations and transactions stay in place. If we try to delete those seperately, EE crashes.

EE crashes in what way?

Are you removing these entities in the database or the UI?

Do you mean you’re deleting the events directly within the DB and then trying to remove registrations from within the UI? If so, that will cause a crash due to the relationships I mentioned above. EE ties everything together using the various relationships between events, registrations, tickets, contacts, messages, prices, promotions, payments etc. Randomly deleting pretty much ANY of the related objects without EE removing the relationship is going to break the UI.


Joost

December 17, 2019 at 4:48 am

Hi Tony,

Some answers for now. I’ve asked my colleague who administers the servers to compile a more detailed overview of the actions we took so far.

  • evidence: no, we do not have evidence that the problem is directly related to the number of events or registrations. If 278 events is a small number for you, the number of registrations will definitely be small in your view, typically between 10 and 250 per event. Approximately 80% of the events do not take registrations
  • crashing: I mean constant 404 errors for all events on the agenda
  • database cleaning: I mean removing entities via de EE UI in the WP backend

In the meanwhile we have been able to mitigate the problem a bit by increasing the RAM of the (dedicated) server once again (now 3GB). From experience I expect this to be a temporary relief. We still need to reset PHP regularly.

I’ll come with more info later.


Tony

  • Support Staff

December 17, 2019 at 5:49 am

evidence: no, we do not have evidence that the problem is directly related to the number of events or registrations. If 278 events is a small number for you, the number of registrations will definitely be small in your view, typically between 10 and 250 per event. Approximately 80% of the events do not take registrations

Just to clarify a little I can’t say 100% that this isn’t the cause, however, it just seems unlikely to me and we’d have multiple reports of this including issues on our own Saas product EventSmart which is powered by Event Espresso.

So because we start trying to digging into database issues, I recommend looking through elsewhere.

crashing: I mean constant 404 errors for all events on the agenda

Ok, thank you.

database cleaning: I mean removing entities via de EE UI in the WP backend

Can you add some more details on this, you mentioned above that EE crashes when you try to remove them, what are you trying to delete and what happens when EE crashes? Any errors?

Do you have a development copy of the site on the same server that I could look at? I’m not about to dig into the production server but if you have (or can create) a copy of the live site to a subdomain etc I’ll happily take a look and see if anything stands out.

(Note, again I can’t say I can fix it as right now we have no idea what is causing the problem, but if I can see anything that stands out it may help)


Joost

December 17, 2019 at 7:08 am

This reply has been marked as private.


Joost

December 17, 2019 at 7:24 am

Hi Tony,

If we try to delete registrations, we get a white screen with a server error (can’t recall what exactly, seemed like a PHP error. Having said that, also this error does not occur always. Now we’ve upgraded the memory to 3GB it works, at least so far.

We have a test server running. We’ll create an admin account for you. I’ll send you the details buh private reply. But note that the test server runs much smoother than the production server. To me, it all indicates to some load-related issue.


Joost

December 17, 2019 at 8:47 am

This reply has been marked as private.


Tony

  • Support Staff

December 19, 2019 at 4:38 pm

Ok, so I went back over your previous thread again to see if anything was missed.

The new branch we are working on isn’t a fix for your current issue and I don’t think it is something we could fix with EE itself. To explain, the above branch helps when permalinks stop working due to another plugin flushing permalinks and forcing WP to lose EE’s rewrite rules. We basically moved when we set those rules to a ‘later’ hook.

None of that would be altered by rebooting PHP, changing PHP versions etc (possibly increasing memory limits, but I’d expect an error if that was the case).

There are a couple of steps above that I’m not clear on and could do with some clarification if possible, I’ll break down each point and respond to all but only some need additional details:

– editing and then saving the event page helped, but not anymore

To be honest, I’m not really sure why this would make any difference.

– upgrading PHP from 5.6 to 7.2 seemed to help for some time

PHP7+ is much more efficient than the previous versions, it can run poor/inefficient code run much better than previous, good/efficient code runs even better.

So if this is due to load, it makes sense that PHP7+ would help.

– resetting the PHP process seemed to help for some time
– restarting the whole server seemed to help sometimes

Maybe a memory leak?

– we replaced the /event/ slug by /activiteiten/ from the EE settings page, since we thought it might have been a conflict with another plugin using the /event/ slug, although we couldn’t think which one that would be. That also seemed to help for a few months.

What exactly did you do here?

Looking on your staging site it looks like you are redirecting all requests to /activiteiten/ to /events/ using the redirect plugin.

I’ll say something I don’t usually say here as people get to fixated on metrics when going down this path, but everything you do on your site has a ‘cost’, be that in RAM, CPU cycles, Queries, Resources. In short everything slows the site down a little but, a little + a little more + a little more can soon mount up to a lot.

‘Pretty’ permalinks are expensive.

Adding additional wildcard permalinks to that, is expensise.

The permalink structure used on your site, is expensive. See this article:

https://dougal.gunters.org/blog/2009/02/04/efficient-wordpress-permalinks/

Things have changed a little since then, but it still worth a read.

– we disabled two other plugins related to redirects: The Redirection plugin and PageLinksTo plugins. that also seemed to help for several weeks, but then it suddenly became worse

Did disabling these also remove the redirects? As in, did you test that a redirect that ONLY worked with the above active no longer works with them disabled?

– adding additional server RAM (we now have 3 GByte) seemed to help for some time

Another +1 for memory leak/load issues.

—-

– we doubled PHP memory in /usr/local/lib/php.ini by memory_limit = 512M

This is where it gets a little confusing and the changes start to make less sense.

So above you set PHP’s memory_limit to be 512M, great, that’s all good. BUT….

– we doubled memory in /wp-config.php by define(‘WP_MEMORY_LIMIT’, ‘256M’);

Backup a second here, WP_MEMORY_LIMIT is a method to set the PHP memory_limit on the server, so you set PHP memory_limit to 512M but then set WP_MEMORY_LIMIT to 256MB.

Why? It’s now less than the memory available by default. Which leads onto this:

– we doubled memory in /wp-includes/default-constants.php as

I’ll prefix this by saying please do not hack up core file, be it WP core files, a plugins core files, a theme’s core files, EE’s core files etc… its unnecessary and causes more problems than it solves.

The changes in this file should not be there, WP_MEMORY_LIMIT is used for this in wp-config.php (that’s the whole point of that constant) and the changes here make the lease sense:

// Define memory limits.
if ( ! defined( ‘WP_MEMORY_LIMIT’ ) ) {
    if ( false === wp_is_ini_value_changeable( ‘memory_limit’ ) ) {
        define( ‘WP_MEMORY_LIMIT’, $current_limit );
    } elseif ( is_multisite() ) {
        define( ‘WP_MEMORY_LIMIT’, ‘128M’ );
    } else {
        define( ‘WP_MEMORY_LIMIT’, ’80M’ );
    }
}

In plain english that first line reads…. if WP_MEMORY_LIMIT is NOT defined do something with default values.

You mentioned above that WP_MEMORY_LIMIT is defined in wp-config.php as it should be, meaning none of the above changes should make a difference. Even if they did, your running out of memory with a memory_limit of 512MB but setting WordPress to use 80MB and it ‘helped’? I don’t understand how that is possible.

WordPress does some checks to see if your current server PHP memory limit is more than WP_MEMORY_LIMIT and attempts to use the higher value of the two so even if the above was changed it should still be using 512MB…. but again, you said this changed helped (even if only for a few hours)… I don’t see how.

(I mean no offence with my reply above, I’m simply pointing out that the changes conflict and don’t really make much sense, especially when they appear to have helped which likely means something was actually the cause/helped in the above situations)

—-

Now having said all that, I’ve been looking at your site at various times and at the time of writing your site has been sitting at a real-time memory usage of 2GB (77% of your total, spiking up to about 85%) and it hasn’t dropped in some time. This leads me to believe the problems does indeed lie with memory. Something on your server is consuming all of the available memory and continues to do so, bumping the memory available helps, until again it is all consumed.

I think you need a SysAdmin to check over your server and get some real-time information on what is consuming all of your memory here, I suspect whatever fixes I could give you aren’t going to fix the real problem just mask it a little.

I’d like to take a quick look at a couple of DB fields on the site, I’ll need to install a plugin to do so (Adminer or similar, or Debug bar and the console add-on) so can pull the values, is that ok?


Joost

December 20, 2019 at 7:56 am

Hi Tony,

Thanks again for your elaborate answer!

I could follow your explanation about the memory and it seems logical to me. We are just guessing and trying many things to keep the site working. And when it works again we think we have found a solution until it stops helping and we start trying new things. We are by no means UNIX or WordPress experts, so we are just trying to find some light in the dark.

The situation at the moment is that we keep getting the 404 errors on the event pages every few hours, not on any other pages. These errors are then consistent until we ‘seem’ to solve it. The easiest fix at the moment appears to be disabling and immediately re-enabling the EE core plugin. After that the 404 errors stay away for a couple of hours.

I tried to install the Adminer tool you mentioned in your last post, but I didn’t succeed. I used this page to do the install: https://www.interserver.net/tips/kb/how-to-install-adminer-in-centos/, but there appear to be a number of errors (some of which I recognized) in the description that prevent me from doing that successfully.

We of course appreciate your offer to have a further look at our server and install that plugin. What would you need to be able to do that? Server root and MySQL root passwords? Anything else?


Tony

  • Support Staff

December 23, 2019 at 3:50 am

I could follow your explanation about the memory and it seems logical to me. We are just guessing and trying many things to keep the site working. And when it works again we think we have found a solution until it stops helping and we start trying new things. We are by no means UNIX or WordPress experts, so we are just trying to find some light in the dark.

Ok, understood, but again my advice is to not hack up any core files.

Doing so for testing may well seem harmless and sometimes can (in rare cases) help point you in the right direction if whoever is modifying the code understands what that code is actually doing and not just randomly making changes and hoping for the best. That’s what appears to have happened above and as mentioned it doesn’t really make sense (again no offence intended).

I get that you may just be trying to find a solution, but hacking up core files (of whatever it is your changing) leaves you in a difficult situation, changes are often forgotten and not reverted, things that would to X now do X+1 or even Y and you end up in a worse situation than you were before.

The situation at the moment is that we keep getting the 404 errors on the event pages every few hours, not on any other pages.

Do you have anything else that adds custom rewrite rules to your site that you can test when doing that or are you using WP pages/posts + EE events?

Point being ‘not on any other pages’ doesn’t really point to an EE issue unless it’s not happening to something else with custom rewrite rules.

These errors are then consistent until we ‘seem’ to solve it. The easiest fix at the moment appears to be disabling and immediately re-enabling the EE core plugin. After that the 404 errors stay away for a couple of hours.

When you’re getting the 404 errors, check your memory consumption and note what it is.

Do the same after you have de-activated EE and again once your have re-activated EE.

I tried to install the Adminer tool you mentioned in your last post, but I didn’t succeed. I used this page to do the install: https://www.interserver.net/tips/kb/how-to-install-adminer-in-centos/, but there appear to be a number of errors (some of which I recognized) in the description that prevent me from doing that successfully.

I don’t need you to install anything for me on the server, I can install a WP plugin and use that, but I wanted to confirm that was ok with your before doing so.

We of course appreciate your offer to have a further look at our server and install that plugin. What would you need to be able to do that? Server root and MySQL root passwords? Anything else?

None of the above, just an ok to install some plugins.

It’s possible, that doing so can bring down the site although it’s not as common now, without FTP access I can’t then de-activate the plugin that causes the error.

Viewing 15 reply threads

The support post ‘Irregular 404 errors – continued’ is closed to new replies.

Have a question about this support post? Create a new support post in our support forums and include a link to this existing support post so we can help you.

Event Espresso