Support

Home Forums Event Espresso Premium CSV download with too many commas

CSV download with too many commas

Posted: November 30, 2020 at 2:15 pm


Oguzhan Altun

November 30, 2020 at 2:15 pm

For some reason, without making any changes, the registrations download broke today. Many lines have too many commas.

Why could this be happening?

There was a custom code added some time ago with your input, could this be broken?
https://eventespresso.com/topic/ee4-registration-download-csv-remove-custom-questions-from-csv-download/


Oguzhan Altun

November 30, 2020 at 2:36 pm

The download:
https://gist.github.com/altuno/59eb260b4f56fbfba0258a47bce71311


Oguzhan Altun

November 30, 2020 at 2:56 pm

It must be one of today’s registrations, because the download was working until yesterday. I can also run a registration download for last month and that works fine.


Oguzhan Altun

December 1, 2020 at 4:43 am

I forgot to mention that there are the following custom functions that impact the CSV download (which has been working perfectly until yesterday’s registrations:
https://gist.github.com/altuno/3a7714f61c2611fa42b2cff99d526ac2


Tony

  • Support Staff

December 1, 2020 at 6:08 am

Ok, so it may be something within the custom code and/or the registration itself so the first thing I would do is try to narrow down the registration(s) causing the CSV to break.

Find your most recent registrations and find the events they are for, then view the registration list for each of those events and export. If ifs a specific registration that should help narrow it down a little as you’ll now know which event it was (assuming that all event registration reports work except one).

Once you’ve narrowed down the registration you start digging into the registrant questions to see if anything stands out there.

You could try removing the above code and commenting out the add_filter() lines to stop them hooking in, or de-activate the plugin they are within to see if the same broken export then works.

If you find a specific event causing it I can take a look at the export and see if anything stands out there if preferred?


Oguzhan Altun

December 1, 2020 at 6:40 am

Yes I found the registration, it’s this one:
https://we.tl/t-koQxr0JA7V

I also found the function that’s creating the problem:
https://gist.github.com/altuno/71b100d90ebaf2b8da256ecdef5f421b

When I remove the function, it works:
https://we.tl/t-5l8hpiVOYe

I think the reason is that we newly added new fields for the users to add their instagram accounts:
https://pasteboard.co/JCUFWW4.jpg
https://pasteboard.co/JCUGaUO.jpg

If that assumption is correct, I’d need your help to modify the code to add these fields to the download file.

Let me know what you think


Tony

  • Support Staff

December 1, 2020 at 7:57 am

Ok, so the first test to do is to change when the above filter runs.

Change this:

add_filter('FHEE__EventEspressoBatchRequest__JobHandlers__RegistrationsReport__reg_csv_array', 'tw_ee_espresso_reg_report_filter_columns_ordered', 10, 2);

To this:

add_filter('FHEE__EventEspressoBatchRequest__JobHandlers__RegistrationsReport__reg_csv_array', 'tw_ee_espresso_reg_report_filter_columns_ordered', 100, 2);

Retest, what happens?


Oguzhan Altun

December 1, 2020 at 9:21 am

Didn’t work, I still get this download:
https://we.tl/t-cdlYIZzGaf


Tony

  • Support Staff

December 2, 2020 at 5:31 am

When I remove the function, it works:
https://we.tl/t-5l8hpiVOYe

That CSV still looks broken to me? Its better than before but Line 1 shows ‘PHOTOBASTEI 2.0’ in the Venue column and then the row below is a column out.

I think the reason is that we newly added new fields for the users to add their instagram accounts

Possibly, its still an odd issue though.

To add those columns to the CSV you can just add:

'ig-name-de',
'ig-text-de'

After this line: https://gist.github.com/altuno/71b100d90ebaf2b8da256ecdef5f421b#file-gistfile1-txt-L46

So it looks something like:

__('Transaction Promotions', 'event_espresso'),
'ig-name-de',
'ig-text-de'

Then you’ll have the ); thats already there to close the array.

I’m not so sure that’s going to fix although it does seem to break on the ig-text-de column.

When you downloaded a CSV does it download when you click the button or does it redirect you to another page with a progress bar and then it downloads?


Oguzhan Altun

December 2, 2020 at 6:10 am

Hello,

It seems like we have the columns in order but the Transaction ID, Attendee ID and the Registration ID fields do not populate after the first line:

https://www.loom.com/share/76fc2c3669e94ffd9592cce81250c5f3

I also tried filtering for October, same issue. Only the first line have these fields, then it stops populating them for the rest of the file.

So not it’s not about a specific registration but something seems broken in how the report populates.


Tony

  • Support Staff

December 2, 2020 at 6:27 am

Did you try what I suggested above?


Oguzhan Altun

December 2, 2020 at 6:40 am

yes sure, that’s why you can now see the instagram handles in the download


Tony

  • Support Staff

December 2, 2020 at 8:58 am

So I spent some time digging into this and I can’t reproduce, however, there have been some changes to EE since you added those snippets which may be affecting this so let’s get those updated and see what happens from there.

The fact that you click the export and wee a ‘progress bar’ means you are using the batch export system (which you should be) and that has different filters available than the ‘old’ system. It checks for any of the old filters in use and applies them into the new system but by doing that EE is having to jump through various hooks so its best to switch to using the new filters (for the most part the only change needed is the filter name itself).

Anywhere you have this: FHEE__EE_Export__report_registrations__reg_csv_array

You can change it to: FHEE__EventEspressoBatchRequest__JobHandlers__RegistrationsReport__reg_csv_array

That’ll switch you over to the new filters (not so scary 😉 ).

Next, each of the CSV rows should have the exact same rows and some of the snippets you are using check for values and/or if the row is for the primary registrant before adding the column. That means some rows may have the columns added and others not, which isn’t a great for the CSV as then commas don’t match (likely the issue here).

Fixing that involves switching the code to always add the column and rather than try to explain that further, I’ve done it here:

https://gist.github.com/Pebblo/d1ab608fac2c68450dab0f458b6baa45

Try that updated code in place of yours and see what happens then.


Oguzhan Altun

December 2, 2020 at 9:57 am

Hi there,

I used the code you sent me, still the same issue.

It must be due to the code section where we set the array of “allowed fields” i guess, because when I remove it, the default fields seem to work.

https://www.loom.com/share/19219cec970148709acb8d84d95e798f


Tony

  • Support Staff

December 2, 2020 at 2:39 pm

It must be due to the code section where we set the array of “allowed fields” i guess, because when I remove it, the default fields seem to work.

Ok, but why all of a sudden and I can’t reproduce so something else must be going on with it.

We’ll need to see what is happening within that function for this and the easiest way to do that is using this little write_log function:

if ( ! function_exists('write_log')) {
   function write_log ( $log )  {
      if ( is_array( $log ) || is_object( $log ) ) {
         error_log( print_r( $log, true ) );
      } else {
         error_log( $log );
      }
   }
}

Add that to the top of your file and then uncomment these lines.

Next, in your sites wp-config.php file, add this:

https://eventespresso.com/wiki/troubleshooting-checklist/#wpdebug

You should already have something like define( 'WP_DEBUG', false ); and that snippet replaces that single line.

Then run an export (I recommend you filter down to the lowest number you can just to make the log easier) and export.

Then check /wp-content/debug.log and search for ‘filtered_csv_row’ you’ll see 2 arrays, one is the registration row and the other is the CSV row. Add them to a\ gist so I can view them please.


Oguzhan Altun

December 3, 2020 at 3:24 am

Here you go:
https://gist.github.com/altuno/fddd17bae5a775396b655285eb56cf05


Tony

  • Support Staff

December 3, 2020 at 3:44 am

Looks like this is due to translations and thats likely why I can’t reproduce.

One more with a slight change to the code to confirm.

Change the write_log call to be:

write_log(
    array(
        'csv_row' => $csv_row,
        'allowed_fields_in_order' => $allowed_fields_in_order,
        'filtered_csv_row' => $filtered_csv_row,
        'registration_db_row' => $registration_db_row
    )
);

Then repeat the above export, please.


Oguzhan Altun

December 3, 2020 at 5:47 am

Here you go:
https://gist.github.com/altuno/d22903dfb640c20a966c8295e747323c


Tony

  • Support Staff

December 3, 2020 at 8:19 am

Yeah, its a translation issue. You said no changes were made on the site, so no plugins updates, nothing?

What are you using for translations on the site?

For your reports the initial request is not translating the CSV headers, see HERE

But then on any additional requests, they are translated to Deutsch, see HERE. You also have some fields translating and others not.

All of the combined is what is causing the problem here as the ‘allowed fields’ are valid for the first request, then they aren’t for any subsequent requests so the fields are missing. It isn’t a case of missing commas but array key fields that don’t match so the values aren’t being set.


Oguzhan Altun

December 3, 2020 at 8:41 am

Hi Tony,

When I said no changes, I meant nothing related to Event Espresso. Of course we update all plugins regularly, including WPML that we use for multi-language. I might have tried to add string translation for some fields a few years ago, then I realized it doesn’t work this way with Event Espresso, so finally ended up changing the name of fields to show both german and English. But this had never created an issue.

Any suggestions how to fix this? We have the same installation on the French website, except we did not yet add the instagram field there, the downloads still work.


Oguzhan Altun

December 3, 2020 at 9:23 am

PS: the wordpress base language of this site is German by the way. The EE4 installation is in English. Maybe that’s the issue?


Tony

  • Support Staff

December 3, 2020 at 9:36 am

When I said no changes, I meant nothing related to Event Espresso.

Any changes on your site can cause changes to any part of your site, so changes made outside of Event Espresso can still affect it.

Of course we update all plugins regularly, including WPML that we use for multi-language.

Ok, so did you update plugins on the site on/around the 30th?

I might have tried to add string translation for some fields a few years ago, then I realized it doesn’t work this way with Event Espresso, so finally ended up changing the name of fields to show both german and English. But this had never created an issue.

Where are you referring to here? The question text fields set on questions? If so that’s not the issue here.

Any suggestions how to fix this? We have the same installation on the French website, except we did not yet add the instagram field there, the downloads still work.

I’m not sure how to fix this and there must be more to this than those fields. To explain a little, as mentioned the function I created basically takes an array of keys/values and then takes an array of ‘allowed’ fields and ‘merges’ them together taking the value from the original when it matches a field in the allowed array.

So then if we look at how that would work, start here:

https://gist.github.com/altuno/d22903dfb640c20a966c8295e747323c#file-debug-13-46-L247-L290

That’s the ‘original array’, it holds the key => value. Notice there that some of the ‘keys’ are translated.

Then we have the allowed fields:

https://gist.github.com/altuno/d22903dfb640c20a966c8295e747323c#file-debug-13-46-L247-L290

Those have no values, but again note they are all translated (that’s from my function).

So then it pulls each element from the allowed list and compares with the original.

“Does the original array have a key ‘Veranstaltung’, yes, use its value”

(This is where things start to go wrong)

“Does the original array have a key ‘Transaktions ID’, no? It’s on the allowed list to set the key but leave it empty as we don’t have a match”

‘Transaktions ID’ is translated in my function, but your csv_row is not, so your getting partial translations and that doesn’t make sense as you should get the same translations for both, so either the transactions load all or none, not some.

PS: the wordpress base language of this site is German by the way. The EE4 installation is in English. Maybe that’s the issue?

EE should just use the language set within the sites general settings (Dashboard -> Settingd -> General -> Site language)


Oguzhan Altun

December 3, 2020 at 10:26 am

Thank you Tony, I opened a ticket with WPML. As you say, when I disable WPML, the download works fine. They’re looking at this ticket right now. If you have any other points that can help them, feel free to add here.

The support post ‘CSV download with too many commas’ is closed to new replies.

Have a question about this support post? Create a new support post in our support forums and include a link to this existing support post so we can help you.

Event Espresso