-
-
Notifications
You must be signed in to change notification settings - Fork 11.4k
Split automated emails into pair for future extensibility #27086
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
EvanHahn
wants to merge
8
commits into
main
Choose a base branch
from
NY-1188
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
41eb205
Create migration to split `automated_emails` into two
EvanHahn 66fe11b
Update table lists with table changes
EvanHahn bf3c4a2
Split `AutomatedEmail` model into `WelcomeEmail{Automation,AutomatedE…
EvanHahn 19cf190
Update `AutomatedEmailRecipient`'s model reference to new table
EvanHahn 29c249a
Update automated emails endpoints to match new data model
EvanHahn 2047e13
Update `MemberWelcomeEmailService` to match new models
EvanHahn 31505e1
Update `MembersAPI` to reflect new models
EvanHahn 8ec646e
Update remaining tests
EvanHahn File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
257 changes: 257 additions & 0 deletions
257
...ons/versions/6.26/2026-04-01-20-46-31-split-automated-emails-into-welcome-email-tables.js
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,257 @@ | ||
| const logging = require('@tryghost/logging'); | ||
| const {commands} = require('../../../schema'); | ||
| const {createNonTransactionalMigration} = require('../../utils'); | ||
| const ObjectId = require('bson-objectid').default; | ||
|
|
||
| const WELCOME_EMAIL_AUTOMATED_EMAILS_AUTOMATION_FK = 'weae_automation_id_foreign'; | ||
| const WELCOME_EMAIL_AUTOMATIONS_FIRST_EMAIL_FK = 'wea_first_email_id_foreign'; | ||
|
|
||
| // Created without the FK on first_welcome_email_automated_email_id to break the | ||
| // circular reference. The FK is added after data population. | ||
| const welcomeEmailAutomationsSpecWithoutFK = { | ||
| id: {type: 'string', maxlength: 24, nullable: false, primary: true}, | ||
| status: {type: 'string', maxlength: 50, nullable: false, defaultTo: 'inactive', validations: {isIn: [['active', 'inactive']]}}, | ||
| name: {type: 'string', maxlength: 191, nullable: false, unique: true}, | ||
| slug: {type: 'string', maxlength: 191, nullable: false, unique: true}, | ||
| first_welcome_email_automated_email_id: {type: 'string', maxlength: 24, nullable: true}, | ||
| created_at: {type: 'dateTime', nullable: false}, | ||
| updated_at: {type: 'dateTime', nullable: true} | ||
| }; | ||
|
|
||
| const welcomeEmailAutomatedEmailsSpec = { | ||
| id: {type: 'string', maxlength: 24, nullable: false, primary: true}, | ||
| welcome_email_automation_id: {type: 'string', maxlength: 24, nullable: false, references: 'welcome_email_automations.id', constraintName: WELCOME_EMAIL_AUTOMATED_EMAILS_AUTOMATION_FK, cascadeDelete: true}, | ||
| subject: {type: 'string', maxlength: 300, nullable: false}, | ||
| lexical: {type: 'text', maxlength: 1000000000, fieldtype: 'long', nullable: true}, | ||
| sender_name: {type: 'string', maxlength: 191, nullable: true}, | ||
| sender_email: {type: 'string', maxlength: 191, nullable: true, validations: {isEmail: true}}, | ||
| sender_reply_to: {type: 'string', maxlength: 191, nullable: true, validations: {isEmail: true}}, | ||
| email_design_setting_id: {type: 'string', maxlength: 24, nullable: false, references: 'email_design_settings.id'}, | ||
| created_at: {type: 'dateTime', nullable: false}, | ||
| updated_at: {type: 'dateTime', nullable: true} | ||
| }; | ||
|
|
||
| const oldAutomatedEmailsSpec = { | ||
| id: {type: 'string', maxlength: 24, nullable: false, primary: true}, | ||
| status: {type: 'string', maxlength: 50, nullable: false, defaultTo: 'inactive', validations: {isIn: [['active', 'inactive']]}}, | ||
| name: {type: 'string', maxlength: 191, nullable: false, unique: true}, | ||
| slug: {type: 'string', maxlength: 191, nullable: false, unique: true}, | ||
| subject: {type: 'string', maxlength: 300, nullable: false}, | ||
| lexical: {type: 'text', maxlength: 1000000000, fieldtype: 'long', nullable: true}, | ||
| sender_name: {type: 'string', maxlength: 191, nullable: true}, | ||
| sender_email: {type: 'string', maxlength: 191, nullable: true, validations: {isEmail: true}}, | ||
| sender_reply_to: {type: 'string', maxlength: 191, nullable: true, validations: {isEmail: true}}, | ||
| email_design_setting_id: {type: 'string', maxlength: 24, nullable: false, references: 'email_design_settings.id'}, | ||
| created_at: {type: 'dateTime', nullable: false}, | ||
| updated_at: {type: 'dateTime', nullable: true}, | ||
| '@@INDEXES@@': [ | ||
| ['slug'], | ||
| ['status'] | ||
| ] | ||
| }; | ||
|
|
||
| module.exports = createNonTransactionalMigration( | ||
| async function up(knex) { | ||
| // 1. Create welcome_email_automations table (without FK to emails, to break circular reference) | ||
| const automationsExists = await knex.schema.hasTable('welcome_email_automations'); | ||
| if (automationsExists) { | ||
| logging.warn('Skipping creating table welcome_email_automations - already exists'); | ||
| } else { | ||
| logging.info('Creating table: welcome_email_automations'); | ||
| await commands.createTable('welcome_email_automations', knex, welcomeEmailAutomationsSpecWithoutFK); | ||
| } | ||
|
|
||
| // 2. Create welcome_email_automated_emails table (with FK + CASCADE to automations) | ||
| const automatedEmailsExists = await knex.schema.hasTable('welcome_email_automated_emails'); | ||
| if (automatedEmailsExists) { | ||
| logging.warn('Skipping creating table welcome_email_automated_emails - already exists'); | ||
| } else { | ||
| logging.info('Creating table: welcome_email_automated_emails'); | ||
| await commands.createTable('welcome_email_automated_emails', knex, welcomeEmailAutomatedEmailsSpec); | ||
| } | ||
|
|
||
| // 3. Copy data from automated_emails into both new tables | ||
| const oldTableExists = await knex.schema.hasTable('automated_emails'); | ||
| if (!oldTableExists) { | ||
| logging.warn('Skipping data migration - automated_emails table does not exist'); | ||
| } else { | ||
| const rows = await knex('automated_emails').select('*'); | ||
| logging.info(`Migrating ${rows.length} rows from automated_emails to new tables`); | ||
|
|
||
| // Only 2 rows exist (free + paid welcome emails), so sequential iteration is fine | ||
| // eslint-disable-next-line no-restricted-syntax | ||
| for (const row of rows) { | ||
| const automationId = ObjectId().toHexString(); | ||
|
|
||
| // Check if already migrated (idempotency) | ||
| const existingAutomation = await knex('welcome_email_automations').where('first_welcome_email_automated_email_id', row.id).first(); | ||
| if (!existingAutomation) { | ||
| // Insert automation first (emails reference automations via FK) | ||
| await knex('welcome_email_automations').insert({ | ||
| id: automationId, | ||
| status: row.status, | ||
| name: row.name, | ||
| slug: row.slug, | ||
| created_at: row.created_at, | ||
| updated_at: row.updated_at | ||
| }); | ||
|
|
||
| await knex('welcome_email_automated_emails').insert({ | ||
| id: row.id, | ||
| welcome_email_automation_id: automationId, | ||
| subject: row.subject, | ||
| lexical: row.lexical, | ||
| sender_name: row.sender_name, | ||
| sender_email: row.sender_email, | ||
| sender_reply_to: row.sender_reply_to, | ||
| email_design_setting_id: row.email_design_setting_id, | ||
| created_at: row.created_at, | ||
| updated_at: row.updated_at | ||
| }); | ||
|
|
||
| // Set back-reference from automation to its first email | ||
| await knex('welcome_email_automations') | ||
| .where('id', automationId) | ||
| .update({first_welcome_email_automated_email_id: row.id}); | ||
| } else { | ||
| logging.warn(`Skipping row for email ${row.id} - already migrated`); | ||
| } | ||
| } | ||
|
|
||
| // 4. Add FK from automations.first_welcome_email_automated_email_id -> emails.id | ||
| logging.info('Adding foreign key from welcome_email_automations to welcome_email_automated_emails'); | ||
| await commands.addForeign({ | ||
| fromTable: 'welcome_email_automations', | ||
| fromColumn: 'first_welcome_email_automated_email_id', | ||
| toTable: 'welcome_email_automated_emails', | ||
| toColumn: 'id', | ||
| constraintName: WELCOME_EMAIL_AUTOMATIONS_FIRST_EMAIL_FK, | ||
| transaction: knex | ||
| }); | ||
|
|
||
| // 5. Drop FK on automated_email_recipients -> automated_emails | ||
| logging.info('Updating foreign key on automated_email_recipients'); | ||
| await commands.dropForeign({ | ||
| fromTable: 'automated_email_recipients', | ||
| fromColumn: 'automated_email_id', | ||
| toTable: 'automated_emails', | ||
| toColumn: 'id', | ||
| transaction: knex | ||
| }); | ||
|
|
||
| // 6. Add FK on automated_email_recipients -> welcome_email_automated_emails | ||
| await commands.addForeign({ | ||
| fromTable: 'automated_email_recipients', | ||
| fromColumn: 'automated_email_id', | ||
| toTable: 'welcome_email_automated_emails', | ||
| toColumn: 'id', | ||
| transaction: knex | ||
| }); | ||
|
|
||
| // 7. Drop the automated_emails table | ||
| logging.info('Dropping table: automated_emails'); | ||
| await commands.deleteTable('automated_emails', knex); | ||
| } | ||
| }, | ||
|
|
||
| async function down(knex) { | ||
| // 1. Recreate automated_emails table | ||
| const oldTableExists = await knex.schema.hasTable('automated_emails'); | ||
| if (oldTableExists) { | ||
| logging.warn('Skipping creating table automated_emails - already exists'); | ||
| } else { | ||
| logging.info('Recreating table: automated_emails'); | ||
| await commands.createTable('automated_emails', knex, oldAutomatedEmailsSpec); | ||
| } | ||
|
|
||
| // 2. Copy data back from new tables to automated_emails | ||
| const automationsExists = await knex.schema.hasTable('welcome_email_automations'); | ||
| const automatedEmailsTableExists = await knex.schema.hasTable('welcome_email_automated_emails'); | ||
|
|
||
| if (automationsExists && automatedEmailsTableExists) { | ||
| const rows = await knex('welcome_email_automations') | ||
| .join( | ||
| 'welcome_email_automated_emails', | ||
| 'welcome_email_automations.first_welcome_email_automated_email_id', | ||
| 'welcome_email_automated_emails.id' | ||
| ) | ||
| .select( | ||
| 'welcome_email_automations.id as automation_id', | ||
| 'welcome_email_automations.status', | ||
| 'welcome_email_automations.name', | ||
| 'welcome_email_automations.slug', | ||
| 'welcome_email_automations.created_at', | ||
| 'welcome_email_automations.updated_at', | ||
| 'welcome_email_automated_emails.id as email_id', | ||
| 'welcome_email_automated_emails.subject', | ||
| 'welcome_email_automated_emails.lexical', | ||
| 'welcome_email_automated_emails.sender_name', | ||
| 'welcome_email_automated_emails.sender_email', | ||
| 'welcome_email_automated_emails.sender_reply_to', | ||
| 'welcome_email_automated_emails.email_design_setting_id' | ||
| ); | ||
| logging.info(`Migrating ${rows.length} rows back to automated_emails`); | ||
|
|
||
| // Only 2 rows exist (free + paid welcome emails), so sequential iteration is fine | ||
| // eslint-disable-next-line no-restricted-syntax | ||
| for (const row of rows) { | ||
| const existing = await knex('automated_emails').where('id', row.email_id).first(); | ||
| if (!existing) { | ||
| await knex('automated_emails').insert({ | ||
| id: row.email_id, | ||
| status: row.status, | ||
| name: row.name, | ||
| slug: row.slug, | ||
| subject: row.subject, | ||
| lexical: row.lexical, | ||
| sender_name: row.sender_name, | ||
| sender_email: row.sender_email, | ||
| sender_reply_to: row.sender_reply_to, | ||
| email_design_setting_id: row.email_design_setting_id, | ||
| created_at: row.created_at, | ||
| updated_at: row.updated_at | ||
| }); | ||
| } else { | ||
| logging.warn(`Skipping automated_emails row ${row.email_id} - already exists`); | ||
| } | ||
| } | ||
|
|
||
| // 3. Update FK on automated_email_recipients | ||
| logging.info('Restoring foreign key on automated_email_recipients'); | ||
| await commands.dropForeign({ | ||
| fromTable: 'automated_email_recipients', | ||
| fromColumn: 'automated_email_id', | ||
| toTable: 'welcome_email_automated_emails', | ||
| toColumn: 'id', | ||
| transaction: knex | ||
| }); | ||
|
|
||
| await commands.addForeign({ | ||
| fromTable: 'automated_email_recipients', | ||
| fromColumn: 'automated_email_id', | ||
| toTable: 'automated_emails', | ||
| toColumn: 'id', | ||
| transaction: knex | ||
| }); | ||
| } else { | ||
| logging.warn('Skipping data migration - new tables do not exist'); | ||
| } | ||
|
|
||
| // 4. Drop FK from welcome_email_automations before deleting referenced table | ||
| logging.info('Dropping foreign key from welcome_email_automations to welcome_email_automated_emails'); | ||
| await commands.dropForeign({ | ||
| fromTable: 'welcome_email_automations', | ||
| fromColumn: 'first_welcome_email_automated_email_id', | ||
| toTable: 'welcome_email_automated_emails', | ||
| toColumn: 'id', | ||
| constraintName: WELCOME_EMAIL_AUTOMATIONS_FIRST_EMAIL_FK, | ||
| transaction: knex | ||
| }); | ||
|
|
||
| // 5. Drop new tables after removing dependent FK | ||
| logging.info('Dropping table: welcome_email_automated_emails'); | ||
| await commands.deleteTable('welcome_email_automated_emails', knex); | ||
| logging.info('Dropping table: welcome_email_automations'); | ||
| await commands.deleteTable('welcome_email_automations', knex); | ||
| } | ||
| ); | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would probably split this migration into a couple of separate migrations. The main smell for me is that this mixes DDL and DML in a single migration. I wouldn't say this is a hard rule (a few existing migrations do violate this), but unless we have a good reason to mix the two, I think it's worth splitting them up. Beyond just being a bit easier to reason about, MySQL has some quirks where e.g. DML operations like creating tables are committed immediately (regardless of any transactions). The risk there is that parts of the migration could succeed, while others fail, leaving the DB in a partially migrated state.
As written today (though I think some of this will change with the data model changes you mentioned), I'd split this into probably 3 migrations: