~dricottone/fmg-timesheets

ref: cab4c59713aeea180456717c05ba8f8d42e0597b fmg-timesheets/parser/timesheet.py -rw-r--r-- 9.5 KiB
Data pipeline error

Some (sub)total rows were being mistaken for actual hour entries.
Previously I cut off all elements ordered after the 'Hours Distribution
by Time Code' marker. This is insufficient because (1) some elements can
be slightly higher and (2) some elements can float to the previous page.

I've fixed (1) by fudging the y-dimension numbers. (2) appears to only
impact a single timesheet (2019-06-15). I've added some more debugging
to help diagnose and communicate this issue.
Fully functional timesheet parser.

The timesheet parser is a complete success. Some minor issues were
ironed out in the XML parser as well.

Next steps: writing to a time series database and beginning analysis.
Implemented time entry extraction; no assert errors!

There is still a major issue ahead of 'structured' data:
Hours data is leaking between entries. There are entries with no hours
at all. There are almost certainly some entries that have hours out of
order.

It will likely be necessary to re-sort all items ahead of processing
based on top then left style attributes. This is going to have the
consequence of invalidating some of the work I've already put into
parsing the data as-is.

Good luck, future me.
Started implementing time entry extraction.

Time entries are now being parsed and validated, though there are
numerous issues still to sort out.

I have a feeling that further development will require passing around
the `top` style attributes in the same way I'm passing around the `left`
style attributes. TBD though.
Significant updates

Wrote time sheet parser that ingests and validates all semi-structured
data. Next step is to interpret left styles as dates, so that hours can
be parsed into a time entry object.

Updated HTML parser to more completely filter out unhelpful data, and to
internally build the array of doubles (data and left style).