30 Commits

Author SHA1 Message Date
1cfc8dbe38
Create EUA and request user access permission
Move the token creation/renewal code into the NordigenClient and remove
external access. The NordigenClient now also takes the DB client in the
constructor.

While creating the unit test, noticed that the nordigen mocks for
downloaded transactions could match the simple transactions mock, thus
helping in the online download command.
Also created the nordigen.NordigenClient mock, with the used methods
mocked and raising a NotImplement when a new endpoint is requested.
2024-01-22 22:01:38 +00:00
420a6cdfaa
Nordigen token generation/refresh logic
Adds a new table, nordigen, with the access and refresh token, along
with their validity.
The Export/Import test would raise an integrety with the use of a real
DB and the export of the transaction IDs, so add a try-except block to
the database session to catch the error and re-raise an ImportError.
2024-01-22 21:59:24 +00:00
d11f753aa0
[Fix] Eager loading for subclasses
and CategoryGroup import/export.
2024-01-22 21:56:34 +00:00
e6622d1e19
Full backup creation and import commands
Using the same logic as the single Export/Import commands, implement the
entire backup command by exporting all the serializable classes into a
single json file.
To select the correct class upon import, save a new property on the
backup json, the class_, which contains the name of the class to be
imported.

Fix the note serialization.
2024-01-22 21:56:17 +00:00
2cf0ba4374
[Export] Remove pickle format
Pickling directly from a DB object will also save DB related attributes,
namely the state of the object. When exporting values from a DB select,
they will be in detached state, which means on the import, the session
will not insert the object back into the DB, it already assumes they are
there.
2024-01-22 21:53:49 +00:00
6574165f8f
Complete export->import cycle
Add the transaction ID to the export of transactions and enable the
import of SplitTransactions. The ID should function as a stable
transaction identifier.
Fix tag import on transactions.
Fix rules import of date for DBs that need datetime.date format.
2024-01-22 21:52:00 +00:00
42d84b02f4
Removes unnecessary FK from Rules __init__
The FK are correctly initialized when the rules are created as part of
the base category/tag.
Also removes the name from the CategorySchedule, same logic applies.
2024-01-22 21:51:49 +00:00
729e15d4e8
Adds ImportFailedError for non-serializable types 2024-01-22 21:51:02 +00:00
638b833c74
ImportCommand and Serializable types
The new command ImportCommand takes a Serializable type, from which it
can call the deserialize method to generate a DB ORM type. The
Serializable interface also declares the serialize method.

(De)serialization moved to the ORM types, due to the inability to
properly use overloading.
Possible improvement for the future is to merge serialization
information on JSONDecoder/Encoder classes.

Adds a MockClient with the in-memory SQLite DB which can be used by
tests.
Most types export/import functionally tested using two DBs and comparing
entries.
2024-01-22 21:49:56 +00:00
21099909c9
Allow Nordigen parameter in Bank constructor
and remove Nordigen FK to Bank from Nordigen constructor. It is auto
generated upon DB insertion.
2024-01-22 21:47:47 +00:00
271130b107
[DB] Rules collection using list instead of set
There pains of using a set as the aggregation of rules overweight the
small advantage. While it would make detecting similar rules faster, the
use of a collection type not inherently supporting by JSON brings some
issues in the serialization.
2024-01-22 21:47:47 +00:00
ec22b5e5bd
[DB][Refactor] Compact the category selector
The `CategorySelector` was possibly added to be incremented with other
attributes. However, since none other that the selector enum is used at
the moment, it is only adding unnecessary cluter.
The category selector value is moved to the parent
`TransactionCategory`.
2024-01-22 21:47:47 +00:00
60c7028f0b
Expand serializer to more types and test 2024-01-22 21:47:47 +00:00
0a0bc1baa0
Fix possible null access
Detected through the typing warning.
2024-01-22 21:47:47 +00:00
01df97ed46
back_populates option on category relationship
Due to the use of the dataclasses mixin on the SQLAlchemy types, a
back_populates creates a RecursiveError when comparing two types. This
occurs because the dataclass will overwrite the __eq__ operator, and it
doesn't know when to stop comparing relationships.

Removing the dataclasses isn't the best approach, since then __init__,
__eq__ and __repr__ methods would have to be added to all types. Thus
the solution was to remove the relationship on the child (on a
one-to-one relationship) from the __eq__ operation, with the use of the
compare parameter.

Took the opportunity to define more logical __init__ methods on the
`Rule` and child classes.
Also revised the parameter options on some DB types.
2024-01-22 21:45:49 +00:00
13c783ca0e
[Refactor] Clean out old DB client class
Swap almost all remaining calls to the old postgresql only DB class with
the new DB client.

Warning! Some operations are currently not implement, such as setting
category schedules and dismantling links.

`update` and `delete` methods added to DB `Client`.
2023-04-30 00:38:15 +01:00
94322ae542
Generalize insert in new Database interface
Move over all inserts on manager.py to new interface.
2023-04-29 20:20:20 +01:00
78ff6faa12
DatabaseSession wrapper for orm.Session
For DB connections that want to keep a session alive, there's a new
`DatabaseSession` class that holds a SQLAlchemy session inside and
offers methods similar to the `Database` class.

The `Database` moves to use the `DatabaseSession` to remove duplicated
code.
2023-04-29 20:20:20 +01:00
ad62317e56
[Fix] Fix sessionless database client
The database `Client` wasn't working correcly when no session was
passed, as the inserted transactions would still be bound to the newly
created session inside.
Creates a copy of the input transactions to insert on the DB.
2023-04-29 20:20:20 +01:00
e7abae0d17
[Refactor] Database client interface changed
`add` method replaced with `insert`.
`insert` and `select` implemented for new database base class.
Database unit test added.

Due to SQLite implementation of the primary key autoinc, the type of the
IDs on the database for SQLite changed to Integer.
https://www.sqlite.org/autoinc.html
2023-04-29 20:20:20 +01:00
761720b712
[Fix] Add typing information to tests
And fix caught errors.
2023-04-23 00:51:22 +01:00
541295ef05
Adds Loader, DatabaseLoader and (Db)Client
Adds unit test to test interfaces.
2023-04-23 00:21:44 +01:00
b1de4d519a
[Refactor] Categorizer now implements Transform
Test adapted for new interface.
Exchanged manual input functionality for throwing an exception.
Removed timer at transformer level.
2023-04-22 22:21:00 +01:00
2c7c527ea9
[Refactor] Creates Tagger transformer
Move the tag rules based transformer to its own class.
Verified with unit tests.
2023-04-22 22:21:00 +01:00
2843c66453
[Refactor] Creates nullifier transformer
Removes nullifier code from Categorizer.
Fixes bug in nullifier rules assertion.
2023-04-22 22:21:00 +01:00
d3c36e30d5
[Refactor] Creates Transform interface
Moves categorizer into transform module.
Puts the categorizer under unit tests.
2023-04-22 22:21:00 +01:00
d6cbb10c47
[Refactor] Move NordigenCredentials to nordigen
Create manager that for now only has the default values from the .env
file.
2023-04-22 22:20:56 +01:00
8710722eb2
[Refactor] Split extractor from Nordigen Client 2023-04-22 22:20:53 +01:00
a3d2d8215e
[Fix] Authentication token missing
Forgot to add the token to the NordigenClient.
Made a test failing, and fixed it.
2023-04-15 15:54:31 +01:00
eb5c1781f0
Get the PSD2 client under testing
Having a comprehension unit testing allows for peace of mind
refactoring.
Changed the PSD2Client constructor and extract method to accept
credentials and banks, respectively.

Create Exception tree for the Extract module.
Adds mocks to the NordigenClient requests' methods.
2023-04-15 13:57:46 +01:00