123 Commits

Author SHA1 Message Date
395576d73f
Move SQLite client to separate file 2022-11-25 19:02:34 +00:00
5f2cb6fcf5
[Fix] Improves logging for bank/creditcard parser
When the credit card was correct but the bank was wrong, it was
difficul to accertain which one was wrong. Not when one fails, it will
also log what are the available options.
2022-11-22 22:09:40 +00:00
9345530f06
[Debug] Retains the downloaded jsons
on the json/ directory.
2022-11-22 22:07:47 +00:00
a364d7b46d
Improves error handling on Nordigen request
by catches an HTTP timeout and retriyng. Also improves logging.
2022-11-22 22:07:47 +00:00
647db5e86f
Fixes convertion to decimal of transactions amount 2022-11-22 22:07:47 +00:00
c41bfb4bdf
Add offset option to bank
A bank had an offset on the date of the transactions...
It also passes the interal Bank type to the converter, to access it's
additional options before converting.
2022-11-22 22:07:04 +00:00
a0d3af715e
Use name instead of institution for bank choice
Also fixes cli method to pass manager ref to the nordigen input.
2022-11-22 21:56:18 +00:00
7e490ffd24
Return to correct database setup
as main development on the nordigen feature ends.
2022-10-09 23:11:24 +01:00
ea3c75245a
Use the DbTransaction type in the manager
The __conform__ can´t generate a tuple from the class, but it is still
worth to use the DB intermediate types for cleaner code. So add tuple()
method the the DBTransaction and use it when writing to the DB.
2022-10-09 23:11:16 +01:00
daed9c5814
Fix the register bank path 2022-10-09 23:11:16 +01:00
98f1730ac2
Add period option to transactions download
The period will only apply after the transactions have been received,
since if sent on the request, credit cards seem to use the value date
instead of the booking date, rendering the feature useless.

Also adds some robustness to the download of the transactions, by adding
a retry in case of timeout.
2022-10-09 23:11:14 +01:00
9300d42527
Adds --all option to download transactions
There is now the possibility to download the transactions for all banks
in the banks table in the DB.
`NordigenInput` parse method fully functional, and entire chain from
downloading to parsing (simple w/ converter) to writing to DB.
DbTransaction type added __conform__ to simplify writes to DB.
Get bank methods added to both `Manager` and `DatabaseClient`.

Warning: csv parser most likely not working at this point.

Issues #16 and #17
2022-10-09 22:44:58 +01:00
cfcc182f35
Fix converters
Python's single dispatch can´t use type aliases and lists of classes.
All converters now work on a class, and lists need to be generated
inplace and call the converter for each item.

Adds the bank class both to the DB schema and the common types.
2022-10-09 22:44:53 +01:00
e2f731a961
Moves input dir up a level
Also imports utils in correspondent __init__.py file.
2022-10-09 22:17:33 +01:00
0a42db8995
Persists banks information and requisition ID
This patch saves the bank information in the DB, in a new table. It also
adds two new CLI commands, register/unregister, so enter the banking
information. (This should later be done internally).

It also adds new types alias for the DB transaction type and new
converters.
Input `transactions` method renamed to `parse`.

Issue #18
2022-10-06 22:22:56 +01:00
ad3fe02e4f
Adds new JSON parser and moves init to Manager
Adds a new abstract `Input` interface for the manager to use different
input handlers with the same methods. The Nordigen class and a new JSON
parser descend from it. The previous csv parser will also eventually.
New converter for list input also added.

Issue #19
2022-10-06 22:22:55 +01:00
c6cfd52b8b
Adds new Manager that will handle components
Move from a direct access to DB by the parsers/categorizers to a middle
layer, which will bring an easier way to have two input alternatives.
This patch starts by instantiating the manager on the cli runnable and
using it for the parser function.

This patch also moves the queries to a different file, so that
introducing new functions on the DB client becomes more manageable and
clearer.

Finally, the new manager will need converters to move from the code type
Transaction to the DB types. This will eventually simplify the code data
model by removing more of its method and leaving it a simple dataclass.

Issue #14
2022-10-06 22:22:54 +01:00
4b5773aa13
Restructes project and adds basic Nordigen API
To improve the code organization, additional subfolders were added into
the pfbudget module.

The barebones of renewing the token and downloading transactions
directly from the banks through the Nordigen API were also added.

Issues #13, #15
2022-10-06 22:22:53 +01:00
2af9b6ebc6
Fix parsing filename for credit cards
If there's no specific parsers for a credit card, use the bank parser.
2022-10-06 22:22:51 +01:00
29b2d9637d
Fix interval option 2022-03-09 22:04:39 +00:00
f8b910b602
Fix filename parsing
For filenames that need more information, such as the same mealcard from
both of us, it was replacing the cc information.
2022-01-01 19:40:56 +00:00
39c534940f
Adds detailed option for report
where all categories expenses are presented.
2021-12-01 19:17:47 +00:00
ab10a5834c
Adds rudementary networth graph 2021-12-01 18:41:07 +00:00
59406c35c1
Splits income into fixed and extra
This will provide differentiation between income that's regular and
stable (long-term contract) and that which is not, such as gigs,
presents, etc.
It presents the information separated in both graphs and reports.
2021-12-01 18:36:45 +00:00
a1f5699b12
Add original and additional comment columns do DB
To have cleared information on manually added transactions to the
database, this patch adds the original column so those added after
the parsing can be marked as such.
Also adds a column to add comments to transaction, when they are not
explicit from the transaction message.
2021-12-01 18:31:25 +00:00
865874f637
Adds group configuration and bugfixes
Now possible to configure each group in the categories.yaml. Only color
added at this moment. The order in which the groups are listed is used
by the graph module and will later be used by the report.

Fixes search for vacations. The negative match is now done by groups,
and the key changes from negative_regex to not_in_groups.
Only updates transactions when any travel is assigned. Was previously
updating all uncategorized transactions that reached the travel search.

Regex match now correctly done lower letters to lower letters.

Transaction __repr__ method added and updated along with __str__, to
provide a more clearer output when manually categorizing.
2021-08-10 23:35:47 +01:00
ac4e14ca35
Orders all select queries by ascending date 2021-08-10 23:16:37 +01:00
2280b60159
Multiple fixes and cleanups
- Remove unnecessary category null check after db select
- Adds multiple try-catch block for misbehaving parsing
- matplotlib.plt tight_layout moved to figure method
- Fixes debit/credit index with different dates
- Check for existing banks transformed to lower chars, to avoid
capitilization problem
2021-07-30 20:19:48 +01:00
1f97a4635a
Clears up program command line menus
Also removes leftover comments.
2021-07-25 20:10:17 +01:00
7cebb33898
Adds '__version__' and completes --version option 2021-07-23 21:22:39 +01:00
0f5123d91e
Adds --save option to graph command
By passing the --save option the create graph is saved to a graph.png
file.
Entire argument dictionary passed to graph function.
2021-07-07 22:22:24 +01:00
5bfb90a547
Remove investments from graph
Investments are not normally reported on the receiving end in
transaction logs, and they introduce too much visual noise in monthly
expense graphs. As such they will later appear in relevant invesment
graphs, but are removed for now from the available choices.
2021-07-07 20:14:03 +01:00
db4397877a
Fix discrete graph
Adds income line to discrete graph.
Keep pfbudget.categories dict import as is to remove name conflicts.
2021-07-07 20:09:56 +01:00
85c0819a44
Adds --category index option
Category argument added to allow the parsing of already categorized
transaction files. The new option takes an integer to indicate the
category index in a transaction row.
2021-07-07 17:07:35 +01:00
ba608093d3
Fix parsing for files w/o bank_cc naming
Fixes code flow for transactions' files with undescriptive filenames.
Adds default indexing to support banks not described in parsers.yaml.
Adds creditcard optional argument.
Command line options now passed as `dict` instead of `Namespace` to
`parse_data`. This will allow for an easier addition of future command
line options.
2021-07-07 17:05:25 +01:00
66262d8dc2
Moves period parser outside of Graph section
`period` ArgumentParser was inside of the Graph section of the
`argparser` function, resulting in confusing logical sequence.
Moved around to the beginning of the function to clarify code.
2021-07-07 16:46:50 +01:00
5af505f962
Catch invalid category options
If there are invalid options in the category.yaml, the program will not
abort and will instead use an empty dictionary.
2021-07-06 19:14:29 +01:00
5f0fe38ba8
Accept a Null category
Null categories can have regex and additional options defined in the
configuration yaml.
2021-07-05 23:38:42 +01:00
c0cc8d5563
Removes obsolete code
Removes `PFState` class and surrounding code.
Removes load/save transactions methods, that were not used since
database introduction.
Removes status command from runnable.py.
2021-07-05 23:07:56 +01:00
37c97453a9
Refactored report module
Bring report up-to-date with sqlite3 database and yaml configuration
files.
Updates report command in runnable.py. Moves date arg parsing to utils
module
Report and graph now share an ArgumentParser with shared date options.
2021-07-05 22:53:03 +01:00
f811b5c711
Name unnamed category groups
Categories without group will now be assigned a No group category
instead of staying blank.
2021-07-05 22:34:51 +01:00
42ab10fd90
Refactored graph module
Bring graph up-to-date with sqlite3 database and yaml files.
Fix sorted transaction query. Removes date as passed parameters.
Updates graph command in runnable.py.
2021-07-04 00:53:07 +01:00
206a828b4a
[Fix] SELECT * now returns list of Transactions 2021-07-04 00:38:29 +01:00
dc6d1d0b02
Add groups to categories in YAML
Groups will later be used to aggregate categories into related groups to
present graphs and reports by type of categories.
2021-07-04 00:36:27 +01:00
9ba7f87a37
[Fix] Add Null to the possible list of categories
The Null category was missing from the available categories when
manually categorizing.
2021-07-03 18:21:12 +01:00
d8c9f3f9b9
[Fix] categorize_data now works w/ Transaction
Updates `categorize_data` to work with the `Transaction` class, which is now
used by the database.
Vacation categorizing will no longer replace previous category assigned.
`DBManager` gained new method to retrieve transactions between two dates
without any category.
Add method to `Transaction` to prepare for category update in database.
Transaction's returns from database queries are checked and assigned in
`if`s with the walrus (`:=`) operator.
2021-07-03 17:47:40 +01:00
1139dff249
Add long --database arg option 2021-07-03 00:51:55 +01:00
fb93f9d4fd
Replaces one-liner function with lambda 2021-07-03 00:23:27 +01:00
703acadaa9
Adds ArgumentParser to contain general arguments
Fix parser func arguments. The database should be passed as an argument,
not used before parsing.
Reordered and added logging to `DBManager` for better recording.
2021-06-30 21:18:06 +01:00
c51d31db47
Fix Postponed Evaluation of Annotations
Adds missing __future__ import.
Moves transaction parser method into Parser class.
2021-06-30 21:12:44 +01:00