Summary
The Bill History segment of the Bills Scraper has been broken for a little while now - if a Bill History item has a null Date, it will break the scraper (which has a type check that does not allow BillHistoryItem.Date to be null) and fail the batch. To fix this, we should make Date optional (and handle that case on the frontend accordingly).
This has not impacted us severely because of the low frequency of updates and the small batch sizes of the scraper, but it also revealed a potential issue in our error reporting that this wasn't surfaced much earlier - the fact that we caught similar bugs in other firebase functions recently makes me suspect that this issue is more recent than not.
One example bill that ran into this issue is https://maple-dev.vercel.app/bills/194/H2572 - there are only three items shown in our Bill History, but there are least five items in the legislature API - one with no date, and one recent item with a Date (that failed to scrape because of the null issue)
Summary
The Bill History segment of the Bills Scraper has been broken for a little while now - if a Bill History item has a
nullDate, it will break the scraper (which has a type check that does not allow BillHistoryItem.Date to be null) and fail the batch. To fix this, we should makeDateoptional (and handle that case on the frontend accordingly).This has not impacted us severely because of the low frequency of updates and the small batch sizes of the scraper, but it also revealed a potential issue in our error reporting that this wasn't surfaced much earlier - the fact that we caught similar bugs in other firebase functions recently makes me suspect that this issue is more recent than not.
One example bill that ran into this issue is https://maple-dev.vercel.app/bills/194/H2572 - there are only three items shown in our Bill History, but there are least five items in the legislature API - one with no date, and one recent item with a Date (that failed to scrape because of the null issue)