We need to re-engineer how we store, and count downloads and other access calls.
Close to 1/3 of our prod. db appears to be the guestbookresponse table now.
Unlike the actionlogrecord table on which we perform a yearly bloodletting, there is no solution for archiving guestbookresponse under the current setup, since we rely on counting the entries in the table for download counts on files and datasets, for display and/or metrics; and because all the downloads and guestbooks data are supposed to be available for viewing and downloading.
I am assuming that it should be doable to add an archiving mechanism. That could be used to delete N years-worth of the oldest records; while preserving the raw counts for display and metrics. And would provide some form of access to them once they are archived. For example, it can be communicated to the users that only the last M years of access records are easily accessible, and the older records need to be retrieved from S3, perhaps 1 year at a time.
I have some ideas that I will add. But I am encouraging everybody to participate/brainstorm.
We need to re-engineer how we store, and count downloads and other access calls.
Close to 1/3 of our prod. db appears to be the guestbookresponse table now.
Unlike the actionlogrecord table on which we perform a yearly bloodletting, there is no solution for archiving guestbookresponse under the current setup, since we rely on counting the entries in the table for download counts on files and datasets, for display and/or metrics; and because all the downloads and guestbooks data are supposed to be available for viewing and downloading.
I am assuming that it should be doable to add an archiving mechanism. That could be used to delete N years-worth of the oldest records; while preserving the raw counts for display and metrics. And would provide some form of access to them once they are archived. For example, it can be communicated to the users that only the last M years of access records are easily accessible, and the older records need to be retrieved from S3, perhaps 1 year at a time.
I have some ideas that I will add. But I am encouraging everybody to participate/brainstorm.