Still towards the TODO list

[DangerousFreedom1984]

Seems like we are in a spiral towards the perfect wallet. Sometimes it feels we are running in circles and sometimes we are getting closer to :p

So, the intent of this post is to propose the minimum acceptable wallet that will get all the basic functions done in a proper way so we could go to a testnet as soon as possible.

I would propose that we break it initially into 5 modules (that could be broken into more modules later):

1. INTERFACE AND INITIALIZATION
   terminal functions, logic flow of wallet,...

2. KEYS AND FILES HANDLING
   generate, restore, rewrite, load, store, write_watch_only_wallet, store_to,
   get_keys_file_data,get_cache_file_data, encrypt_keys, decrypt_keys,
   save_to_file, load_from_file

3. TRANSACTION CREATION
   commit_tx, save_tx, sign_tx, load_unsigned_tx, load_tx, create_transactions_*,

4. TRANSACTION POST PROCESSING
   balance, unlocked_balance, get_transfers, get_payments, get_unconfirmed,
   get_tx_key,get_tx_proof,get_spend_proof, get_address_book,

5. RPC / ENGINE (IMPLEMENTATION) CONNECTION
   update_pool_state, get_base_fee,

The idea is to create properly the most basic functions to achieve the following milestones:

1. -M1: terminal working with essential commands available
   -M2: help
   -M3: screen locker

2. -M1: functions to store in a file a jamtis_mock_keys struct and load it
   -M2: encrypt/decrypt files properly
   -M3: other functions

3. -M1: make a transaction to an address
   -M2: add options to transactions
   -M3: ...

4. -M1: visualize transactions made and received
   -M2: get info about enotes
   -M3: transaction proofs

5. -M1: connection to ledger to add and receive blocks and txs
   -M2: estimate fees
   -M3: ...

The line of thought being: when all M1 of all modules are achieved, then we would have the most basic proper functional wallet.

All of these modules have already many or most of the things done but not yet in a clean and coherent way. So I guess we are missing a bit of work coordination and consensus about the most basic milestones and acceptable code.

So, is it a good idea to organize and track the work like that? If so, we could better specify the milestones and organize who will be doing what. Would you change something?

[rbrunner7]

Thanks for writing this issue. Something like a "todo list", even if only a partial one, slowly becomes pressing for sure.

It's quite interesting to me how the ways of the two of us to look at this project, and how to possibly go forward with, differ. I think I mentioned a bit about my ideas in this regard already in a workgroup meeting, and certainly in one of our direct talks, but I think it may be useful to write it down here in more detail, also based on the insights gained in the meantime, so people have a chance to compare and comment:

I would free my mind from all thinking about longer-term goals and fully concentrate my sights on the simplest possible goal that would deserve the term "wallet" in any reasonable sense. Reason, in short: Already that simplest possible goal is big and hard to achieve, I wouldn't want to make it any more difficult if there is a way. Another reason: Over the course of implementing that, despite being very simple, almost trivial, I will gain many insights that will tell me how to go forward from there.

Insights that are hard to get or even impossible to get if I don't implement something first. I expect a number of surprises, like I encountered them when I used exactly this approach for several of my private free-time programming projects.

So, what would I build? A program that can take a random single enote that I own, make a transaction out of that, and submit it. Then take another random single enote and make another transaction. Tomorrow again with more enotes after I switched off my computer in the meantime.

Single enote to go without a proper enote selection module for the time being. Random enote because I don't care about the transferred amount yet.

What does my program need to handle? Keys of course. And it must be able to store them, so I can tomorrow continue with the same keys.

Enotes. I need a component to handle my list of owned enotes. And also store and load them. And keeping track which are spent and which not yet of course.

Do I need to store more things beside keys and enotes to start? I claim "no". Not for being able to do what I described. That will give one trivial wallet file! But still, that will be progress, for example you will have to validate and use @UkoeHB's approach of serializing Seraphis-related objects along the way.

The enotes have to come from somewhere. So I attach @j-berman's scanner to get me some legacy enotes that I wrote into the blockchain using any of the current wallet apps.

And I need some rudimentary way to talk to a daemon. Should be possible with direct calls right in the places where I need them, i.e. without a full daemon communication module yet. Parameters all standard, local daemon only. Nothing to configure. That comes later.

What do I need to operate that program? Really not much. When it starts, it asks me whether to load an existing wallet file, or create a new wallet. From then on it only waits for futher keypresses to build and submit one transaction per keypress, and makes some log output about what's happening on the way.

Now, I don't know whether this all sounds pretty dumb to readers, whether I already lost them after 10 lines because not being interesting enough, but whatever: That's the way I would do it if I was alone and had absolutely free reign. No doubt. And I am convinced already this "toy wallet" would be quite a milestone with all kinds of positive effects on the project.

[DangerousFreedom1984]

In this case I don't think your answer differs from the intent of my approach. I was just trying to be more specific.

[ghostway0]

@rbrunner7 that does sound like what I'd do. But I think dangerousfreedom's way isn't that different...
I have some time to work on it today. Shouldn't that struct be moved out of mocks into some kind of a seraphis_common directory?
Or at least, eventually moved out of mocks? Also, is serialization changing or should I just add BEGIN_SERIALIZEs?

[DangerousFreedom1984]

Right on the point. I would first do it using the mock there. It will have to change to another place eventually but then the discussion about the wallet tiers will come along too. Regarding the serialization, I don't know if we have consensus about the best way to do it. I hope you will find the answers and push the discussion forward. If helpful, my first attempt to tackle this problem can be found here and here. Let us know your thoughts and thanks for looking at it.

[rbrunner7]

It's slowly dawning on me how difficult it will probably be to get this going, with a group of people that don't know each other well yet, and with doing everything in writing. People acting in good faith, trying their best, but difficult circumstances. It will certainly get better after managing to really get moving, but now I feel it's very hard.

I will gladly try to answer questions like here, but please keep in mind that I will be less and less able to provide any answers as time and work progresses. With quite some restraint I do not go into detailed designing and programming of the Seraphis wallet. It's very tempting, but I will directly walk into overwork and burnout if I do. "Managing" the whole venture already eats almost all of my time.

So it happens that unfortunately I don't have much to offer regarding the design and the implementation of the key container.

But well, we have a number of design questions on the table, and seems to me the only sensible way is to methodically and thoroughly study them one by one until good answers can be found, right? I have a personal saying here: "Sometimes only really understanding something helps".

(If I should come over in the following like some "Mr. Know It All" or like somebody who tries to boss people around or somebody who thinks everybody else is a bit stupid that's a misunderstanding. Just honestly trying to be helpful and also giving people some hints about the way how I "tick" to make future communication a bit less hard.)

There are in the current library two structs with key material already. @DangerousFreedom1984 used them in the code he linked. Is that the right approach? I don't know. How to find out? Reading the library, checking where and how these structs are used, how important and fundamental they are.

I see two possible outcomes of such an investigation right now, but maybe there are more: They really are only closely connected to mocking, in which case we probably should define our own, or they are quite fundamental and important, in which case we should probably use them but on the same time open a discussion with @UkoeHB whether names with "mock" in it are still appropriate in this particular case or whether we should rename them.

About serializing. In the Monero codebase we have two kinds of serialization that are widely used and with broad support, binary_archive and that "KV" stuff. Which do we take? Good question. I always wondered why for things that go into the wallet .keys files currently KV (or even RapidJason) and not binary is used. Binary is faster, more compact, as far as I know almost universally used over the Monero codebase if one has to write into files, but maybe wallet files are in some way special? Or KV is as an exception a good idea here for some other reason? Interesting research job.

Certainly feel free to reach beyond our little group into the "broader" Monero dev community and for example throw any specific questions about wallet serialization approaches into the dev IRC channel or Matrix room.