Contact role updates/deletions not always syncing correctly to Salesforce

[svogt0511]

Purpose

Purpose is to keep the provider/contact information in sync between the database and salesforce

Related to:

• Contact role updates/deletions not always syncing correctly to Salesforce datacite#2207
• https://github.com/datacite/product-backlog/issues/859

Additional pull request required for this fix: datacite/mastino#335

Additional issue regarding the salesforce sync tasks to be discussed after this is released: (https://github.com/datacite/product-backlog/issues/895) This would be for the provider and contact sync jobs. The client one stays in place because it updates the repository doi count in salesforce which is not information pushed to salesforce as part of the client update.

Approach

The first task was to keep the provider/contact information in sync in the DB. Some information is shared between the provider and its contacts. We needed to add some code to both the API provider endpoints (create/update/delete), the contact endpoints (create/update/delete), and their models to enforce synchronization of this data when changes are made.

The second task was to ensure that changes are pushed to salesforce. To this end, the calls to send updated information (send_contact_export_message and send_provider_export_message) were moved from the indexable.rb after_commit to the provider and contact models, to ensure that changes are pushed after all information has been made consistent.

The last task was to ensure that the pathway for updates from lupo to salesforce is working. This includes testing which can be done with rake tasks to push providers and contacts from lupo to salesforce. Initially this seemed to work, but ongoing inconsistent delivery of messages to salesforce led to the discovery that the salesforce-api_runner was getting 10 messages per block and only processing the first. The pull request on mastino mentioned above should take care of this issue.

We need a remediation script to be run after the release that ensures the provider/contact information is consistent in the DB, and that updated information is pushed to salesforce. All that needs to happen is to run an update on each provider. That should take care of the necessary updates.

Manual testing can be done in staging using either postman or fabrica. To ensure that the updates are pushed to salesforce, you can log into the salesforce sandbox using the info@datacite.org.sync account, or your own account if you have one. Kelly outlined some test scenarios in the issue mentioned above.

Open Questions and Pre-Merge TODOs

• Write a remediation script - a rake task. Test it in staging.
• Additional test(s) to the test suite.

Then:

• Manual testing - Either through Fabrica or postman (create/update/delete providers, create/update/delete contacts, assign provider roles to contacts. Verify this information is properly transmitted to salesforce which can be done by logging into salesforce and looking at contacts and providers in the salesforce sandbox. (use info@datacite.org.sync account)
• Review and release mastino - to add the fix to salesforce-api_runner (Fix salesforce-api-runner.js to handle a message block mastino#335).
• Review and merge to lupo (this PR). Leave in staging for a few days for additional testing.
• Release lupo (this PR).
• Run the remediation script after the lupo release (in test and production).
• Discuss and possibly stop running the salesforce sync jobs for providers and contacts (Issue #895). (I.e., Does the real time forwarding of provider/contact updates handle all of the information that needs to be forwarded to salesforce?). Clients/repositories need to keep running because they need a few extra pieces of information that are not forwarded through lupo. I am going to add this as a separate issue because we don't need this to be resolved before releasing this PR.

Learning

Types of changes

• Bug fix (non-breaking change which fixes an issue)

• New feature (non-breaking change which adds functionality)

• Breaking change (fix or feature that would cause existing functionality to change)

Reviewer, please remember our guidelines:

• Be humble in the language and feedback you give, ask don't tell.
• Consider using positive language as opposed to neutral when offering feedback. This is to avoid the negative bias that can occur with neutral language appearing negative.
• Offer suggestions on how to improve code e.g. simplification or expanding clarity.
• Ensure you give reasons for the changes you are proposing.

[jrhoads]

Seems like we would want to remove this

[jrhoads]

Seems like we would want to remove this

[jrhoads]

remove commented puts

[svogt0511]

Thanks. I thought that I had already cleaned those up.