218 Commits

Author SHA1 Message Date
f721e6a910
[Refactor] Decouple CLI arguments from Manager
The Manager doesn't need to know that it was called from the CLI, so it
now is initialized with the database and performs an action, based on
the operation it receives and its parameters.

The work isn't finished, some Manager actions are still based on the CLI
arguments.

The CLI logic and creation of parameters to pass to the manager have
been moved to the __main__.py file, which brings it to line to the
program being called as a package from the command line.
2022-12-09 19:43:09 +00:00
471331ffc9
Categories rules for rule-based categorization
This patch extends the categories_rules with a rule for each field of a
transaction.

It also changes the ORM classes to behave as dataclasses again.
2022-12-09 19:38:12 +00:00
63d6a2aab7
Finishes category schedule implementation 2022-12-08 20:15:48 +00:00
f09b328c66
Package now run as python3 -m pfbudget
Moves in line in the normal way python packages are run.
https://docs.python.org/3/library/__main__.html
2022-12-08 20:11:33 +00:00
6f97b883fb
Add weekly option on the schedule period 2022-12-08 16:44:07 +00:00
72974c90aa
Operations refactor
Unify the manager operations under one enum.
Start cleaning up the argparser structure to easily add new operations.
2022-12-08 16:15:53 +00:00
d11bc6df1d
Adds category recurring possibility 2022-12-08 13:31:31 +00:00
d409038072
Adds category selector column
This column indicates where has the category came from.
2022-12-08 01:22:26 +00:00
9d33df78a8
Adds command line option to add/remove categories
Implements the argument parser, the manager logic and the DB client
methods.
Encapsulates the DbClient connection under the _db attribute on the
manager.

Adds verbose option to enable ORM increased logging.
2022-12-08 00:25:56 +00:00
882a77d24c
[Refactor] CLI argparser passes options to Manager
Move all BL to the manager. The ArgParser now only parses the CLI
arguments and creates a command which contains which command was run. In
turn, this information is passed to the manager, which will run the
appropriate business logic.

This will make it easier to add new options, separating the parsing of
the CLI options from the implementation of the logic. It also simplifies
any future effort in adding a different input (e.g. GUI).

Warning: some function were commented out, this is only a tracer bullet.
2022-12-04 17:48:28 +00:00
be67612f67
Introduces categorizer that works on ORM classes
Categorizer will work directly on ORM classes, which will cleanup the
code, since changes will automatically be persisted when change the
objects.

Adds wrapper session class inside the DbClient for the manager to use.
The manager will have to have some DB session knowledge, which adds some
unfortunate coupling.

Removes some unnecessary relations between tables that were added by
mistake.

category CLI option now uses the manager.
2022-12-04 16:13:05 +00:00
78e545589d
Adds category groups 2022-12-04 15:14:24 +00:00
b8142f4f99
Add available categories table 2022-12-04 11:19:21 +00:00
e379d77995
Nordigen -> PostgreSQL path completed
Can now download from all banks registered on the banks/nordigen table
and write to the PostgresSQL DB (or any DB, since we're now agnostic).

Commented out most of the managers functions until the integration with
the new DB client is complete.
Set Optional relationships. Remove the DB types as dataclasses, it only
increased the verbosity of the types w/o much going for it.
Change the name Original to Transaction, since the type is the
placeholder for the rest of the transaction information.
2022-12-04 00:02:45 +00:00
91514f71b1
Adds relationships and CASCADE on deletes 2022-12-03 16:57:27 +00:00
246c948d76
All meal card to account type enum 2022-12-03 16:13:26 +00:00
0d22b02b3f
Adds alembic tool for DB versioning
Alembic is a lightweight database migration tool for usage with the
SQLAlchemy Database Toolkit for Python.
https://alembic.sqlalchemy.org/en/latest/index.html

Adds first version of DB schema.
2022-12-03 16:13:23 +00:00
395576d73f
Move SQLite client to separate file 2022-11-25 19:02:34 +00:00
5957242b83
Merge branch 'feature/nordigen' 0.2 2022-11-22 22:12:32 +00:00
5f2cb6fcf5
[Fix] Improves logging for bank/creditcard parser
When the credit card was correct but the bank was wrong, it was
difficul to accertain which one was wrong. Not when one fails, it will
also log what are the available options.
2022-11-22 22:09:40 +00:00
9345530f06
[Debug] Retains the downloaded jsons
on the json/ directory.
2022-11-22 22:07:47 +00:00
a364d7b46d
Improves error handling on Nordigen request
by catches an HTTP timeout and retriyng. Also improves logging.
2022-11-22 22:07:47 +00:00
647db5e86f
Fixes convertion to decimal of transactions amount 2022-11-22 22:07:47 +00:00
c41bfb4bdf
Add offset option to bank
A bank had an offset on the date of the transactions...
It also passes the interal Bank type to the converter, to access it's
additional options before converting.
2022-11-22 22:07:04 +00:00
a0d3af715e
Use name instead of institution for bank choice
Also fixes cli method to pass manager ref to the nordigen input.
2022-11-22 21:56:18 +00:00
e3756fa442
Update requirements.txt 2022-10-10 21:21:15 +01:00
7e490ffd24
Return to correct database setup
as main development on the nordigen feature ends.
2022-10-09 23:11:24 +01:00
ea3c75245a
Use the DbTransaction type in the manager
The __conform__ can´t generate a tuple from the class, but it is still
worth to use the DB intermediate types for cleaner code. So add tuple()
method the the DBTransaction and use it when writing to the DB.
2022-10-09 23:11:16 +01:00
daed9c5814
Fix the register bank path 2022-10-09 23:11:16 +01:00
98f1730ac2
Add period option to transactions download
The period will only apply after the transactions have been received,
since if sent on the request, credit cards seem to use the value date
instead of the booking date, rendering the feature useless.

Also adds some robustness to the download of the transactions, by adding
a retry in case of timeout.
2022-10-09 23:11:14 +01:00
9300d42527
Adds --all option to download transactions
There is now the possibility to download the transactions for all banks
in the banks table in the DB.
`NordigenInput` parse method fully functional, and entire chain from
downloading to parsing (simple w/ converter) to writing to DB.
DbTransaction type added __conform__ to simplify writes to DB.
Get bank methods added to both `Manager` and `DatabaseClient`.

Warning: csv parser most likely not working at this point.

Issues #16 and #17
2022-10-09 22:44:58 +01:00
cfcc182f35
Fix converters
Python's single dispatch can´t use type aliases and lists of classes.
All converters now work on a class, and lists need to be generated
inplace and call the converter for each item.

Adds the bank class both to the DB schema and the common types.
2022-10-09 22:44:53 +01:00
e2f731a961
Moves input dir up a level
Also imports utils in correspondent __init__.py file.
2022-10-09 22:17:33 +01:00
0a42db8995
Persists banks information and requisition ID
This patch saves the bank information in the DB, in a new table. It also
adds two new CLI commands, register/unregister, so enter the banking
information. (This should later be done internally).

It also adds new types alias for the DB transaction type and new
converters.
Input `transactions` method renamed to `parse`.

Issue #18
2022-10-06 22:22:56 +01:00
ad3fe02e4f
Adds new JSON parser and moves init to Manager
Adds a new abstract `Input` interface for the manager to use different
input handlers with the same methods. The Nordigen class and a new JSON
parser descend from it. The previous csv parser will also eventually.
New converter for list input also added.

Issue #19
2022-10-06 22:22:55 +01:00
c6cfd52b8b
Adds new Manager that will handle components
Move from a direct access to DB by the parsers/categorizers to a middle
layer, which will bring an easier way to have two input alternatives.
This patch starts by instantiating the manager on the cli runnable and
using it for the parser function.

This patch also moves the queries to a different file, so that
introducing new functions on the DB client becomes more manageable and
clearer.

Finally, the new manager will need converters to move from the code type
Transaction to the DB types. This will eventually simplify the code data
model by removing more of its method and leaving it a simple dataclass.

Issue #14
2022-10-06 22:22:54 +01:00
4b5773aa13
Restructes project and adds basic Nordigen API
To improve the code organization, additional subfolders were added into
the pfbudget module.

The barebones of renewing the token and downloading transactions
directly from the banks through the Nordigen API were also added.

Issues #13, #15
2022-10-06 22:22:53 +01:00
2af9b6ebc6
Fix parsing filename for credit cards
If there's no specific parsers for a credit card, use the bank parser.
2022-10-06 22:22:51 +01:00
29b2d9637d
Fix interval option 0.1 2022-03-09 22:04:39 +00:00
3ef60a96b7
Add TODO 2022-03-09 22:02:55 +00:00
f8b910b602
Fix filename parsing
For filenames that need more information, such as the same mealcard from
both of us, it was replacing the cc information.
2022-01-01 19:40:56 +00:00
39c534940f
Adds detailed option for report
where all categories expenses are presented.
2021-12-01 19:17:47 +00:00
ab10a5834c
Adds rudementary networth graph 2021-12-01 18:41:07 +00:00
59406c35c1
Splits income into fixed and extra
This will provide differentiation between income that's regular and
stable (long-term contract) and that which is not, such as gigs,
presents, etc.
It presents the information separated in both graphs and reports.
2021-12-01 18:36:45 +00:00
a1f5699b12
Add original and additional comment columns do DB
To have cleared information on manually added transactions to the
database, this patch adds the original column so those added after
the parsing can be marked as such.
Also adds a column to add comments to transaction, when they are not
explicit from the transaction message.
2021-12-01 18:31:25 +00:00
865874f637
Adds group configuration and bugfixes
Now possible to configure each group in the categories.yaml. Only color
added at this moment. The order in which the groups are listed is used
by the graph module and will later be used by the report.

Fixes search for vacations. The negative match is now done by groups,
and the key changes from negative_regex to not_in_groups.
Only updates transactions when any travel is assigned. Was previously
updating all uncategorized transactions that reached the travel search.

Regex match now correctly done lower letters to lower letters.

Transaction __repr__ method added and updated along with __str__, to
provide a more clearer output when manually categorizing.
2021-08-10 23:35:47 +01:00
ac4e14ca35
Orders all select queries by ascending date 2021-08-10 23:16:37 +01:00
9198911c5a
Bring flake8 setup up-to-speed with black 2021-07-30 20:23:57 +01:00
2280b60159
Multiple fixes and cleanups
- Remove unnecessary category null check after db select
- Adds multiple try-catch block for misbehaving parsing
- matplotlib.plt tight_layout moved to figure method
- Fixes debit/credit index with different dates
- Check for existing banks transformed to lower chars, to avoid
capitilization problem
2021-07-30 20:19:48 +01:00
1f97a4635a
Clears up program command line menus
Also removes leftover comments.
2021-07-25 20:10:17 +01:00