192 Commits

Author SHA1 Message Date
78ff6faa12
DatabaseSession wrapper for orm.Session
For DB connections that want to keep a session alive, there's a new
`DatabaseSession` class that holds a SQLAlchemy session inside and
offers methods similar to the `Database` class.

The `Database` moves to use the `DatabaseSession` to remove duplicated
code.
2023-04-29 20:20:20 +01:00
9f39836083
[Refactor] Export ops move to Client interface
Introduce new `Manager` parameter database to use new DB client API.
Fix TransactionCategory format.
Fix Client kwargs type.
2023-04-29 20:20:20 +01:00
ad62317e56
[Fix] Fix sessionless database client
The database `Client` wasn't working correcly when no session was
passed, as the inserted transactions would still be bound to the newly
created session inside.
Creates a copy of the input transactions to insert on the DB.
2023-04-29 20:20:20 +01:00
e7abae0d17
[Refactor] Database client interface changed
`add` method replaced with `insert`.
`insert` and `select` implemented for new database base class.
Database unit test added.

Due to SQLite implementation of the primary key autoinc, the type of the
IDs on the database for SQLite changed to Integer.
https://www.sqlite.org/autoinc.html
2023-04-29 20:20:20 +01:00
9c7c06c181
[DB] Schema unified on pfbudget 2023-04-29 20:20:16 +01:00
d6779b40ea
Alembic added to poetry 2023-04-23 16:37:11 +01:00
761720b712
[Fix] Add typing information to tests
And fix caught errors.
2023-04-23 00:51:22 +01:00
541295ef05
Adds Loader, DatabaseLoader and (Db)Client
Adds unit test to test interfaces.
2023-04-23 00:21:44 +01:00
14131f50f9
[Refactor] Rename the PostgreSQL file 2023-04-23 00:00:45 +01:00
c4d2d5a378
Install the psycopg binary along with the package 2023-04-22 23:55:07 +01:00
ef9ca1d1ea
Adds the mypy linter 2023-04-22 22:21:00 +01:00
b1de4d519a
[Refactor] Categorizer now implements Transform
Test adapted for new interface.
Exchanged manual input functionality for throwing an exception.
Removed timer at transformer level.
2023-04-22 22:21:00 +01:00
2c7c527ea9
[Refactor] Creates Tagger transformer
Move the tag rules based transformer to its own class.
Verified with unit tests.
2023-04-22 22:21:00 +01:00
2843c66453
[Refactor] Creates nullifier transformer
Removes nullifier code from Categorizer.
Fixes bug in nullifier rules assertion.
2023-04-22 22:21:00 +01:00
d3c36e30d5
[Refactor] Creates Transform interface
Moves categorizer into transform module.
Puts the categorizer under unit tests.
2023-04-22 22:21:00 +01:00
d6cbb10c47
[Refactor] Move NordigenCredentials to nordigen
Create manager that for now only has the default values from the .env
file.
2023-04-22 22:20:56 +01:00
8710722eb2
[Refactor] Split extractor from Nordigen Client 2023-04-22 22:20:53 +01:00
a3d2d8215e
[Fix] Authentication token missing
Forgot to add the token to the NordigenClient.
Made a test failing, and fixed it.
2023-04-15 15:54:31 +01:00
eb5c1781f0
Get the PSD2 client under testing
Having a comprehension unit testing allows for peace of mind
refactoring.
Changed the PSD2Client constructor and extract method to accept
credentials and banks, respectively.

Create Exception tree for the Extract module.
Adds mocks to the NordigenClient requests' methods.
2023-04-15 13:57:46 +01:00
184628dcfc
Rename Nordigen to PSD2 2023-04-13 18:55:33 +01:00
26cf921a6f
Fix flake8 warnings 2023-04-13 18:38:07 +01:00
d7bdafa62a
Rename input -> extract
Trying to follow the common ETL pattern naming.
2023-04-12 19:51:10 +01:00
88a29bdf83
Update all dependencies and add flake8 2023-04-12 19:50:24 +01:00
2ad5f8171d
Move to poetry as the dependency management tool 2023-04-12 19:13:19 +01:00
84fa87958a
Change MIT for GNU GPL v3.0 license 2023-04-12 19:12:03 +01:00
ddb02b33f9
Merge branch 'feature/postgresql'
The graphs and reports are broken on this commit.
2023-04-12 18:25:16 +01:00
5af05c2fa4
Print the transaction ID 2023-04-12 18:23:24 +01:00
2cbf00a939
CLI now allow --no-nulls flag on categorize
Null categorization, which is a major time slog, may now be disabled.
2023-04-12 18:20:57 +01:00
6b26651d22
Option eua now takes id instead of bank name 2023-04-12 18:19:24 +01:00
ed2dda63e9
Allows using rules for the nullying step 2023-02-23 23:24:01 +00:00
1a774e3769
Adds get all transactions operation 2023-02-23 23:23:19 +00:00
dd724b6c28
Export in .csv
Importing is not supported, since there's no way to represent a Null
field in .csv
2023-02-23 23:21:54 +00:00
6f68d971ee
Clear up forge/dismantle logic 2023-02-11 22:48:04 +00:00
f7df033d58
Add start date rule
Rename date to end.
2023-02-11 22:46:41 +00:00
23eb2c80bd
[Interactive] Defines an Interactive class
Affords a cleaner coding over the function.
Renames the runnable.py into what it actually is, the argparser.py.
2023-01-30 22:24:23 +00:00
7453ffbd3a
[Interactive] Adds new format for tags
Now tags can be defined along side categories by starting with :.
E.g. eating out:work:today will classify the transaction with the eating
out category and work and today tags.
2023-01-29 23:48:48 +00:00
da348c4ffb
[Fix] Splitted categories were not being commited 2023-01-29 23:48:27 +00:00
2da721d53c
[Interactive] Adds skip command 2023-01-29 23:48:22 +00:00
f943374ade
[Fix] Splitted transactions do not have category
Adds uncategorized method to the DB client to retrieve transactions w/o
a category AND not splitted.
2023-01-29 23:39:46 +00:00
abff76ad4e
[Interactive] Finish split command 2023-01-29 21:26:22 +00:00
e0fc310ef6
[Fix] Add split default 2023-01-29 21:16:05 +00:00
13709b7a04
[Interactive] Adds new tag 2023-01-29 21:14:49 +00:00
fd24ac3318
Remove unnecessary __init__.pys
and import using submodules in __main__.py
2023-01-23 23:52:20 +00:00
36e7f84bd9
Interactive categorization moved to __main__.py
It didn't make sense to have it inside the manager, it should only be
used to process commands and its paramaters.
2023-01-23 23:31:38 +00:00
5235fcdfc3
Adds business logic for split transaction
Moves split member to be part of parent class and removes it from the
init.
2023-01-23 23:03:42 +00:00
ace5195164
Move selector table back to transactions schema
Change table name originals to transactions and tags to tagged.
2023-01-23 19:36:24 +00:00
ae0a195069
Split tables per different schemas 2023-01-23 19:36:07 +00:00
8760f5a0a4
Export/Import categories and groups 2023-01-23 00:06:36 +00:00
dd0aaa01b8
Export/import for banks 2023-01-22 23:42:32 +00:00
fd6793b4f4
Turned on type checking
and as a result, had to fix a LOT of minor potential future issue.
It also reorders and clears unused imports.

When exporting transactions, it will sort by date.
2023-01-22 20:44:05 +00:00