[Software] -- Electroneum Daemon sucks -- ideas?

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

funkywizard

mmm.... bandwidth.
Jan 15, 2017
848
402
63
USA
ioflood.com
Heya.

Some of you may have noticed that cryptopia has had wallet issues recently with ETN. But, they're not the only ones. Tradeogre and qryptos recently started trading ETN as well, and their wallets have had sporadic issues as well.

As some background, I'm self-pooling electroneum, as well as trying to buy hash on nicehash when it's cheap. When buying hashpower, I keep a close eye on the health of my pool and daemon(s). I can tell you, electroneum is pretty terrible for this. I'll have 30 peers one minute, and 5 the next. When I have 30 peers, you'll see half of them 1 - 2 blocks behind for 15+ seconds after a new block comes out, sometimes more than 1 minute behind. The target block time is only 1 minute! So any nodes that are behind by 30 seconds are mining the wrong block half the time, and behind by 1 minute they're mining the wrong block most of the time.

I'm running a cluster of daemons, and I'll see my primary daemon pick up a new block because 1 out of 30 peers has it. Half the time this is an external peer -- none of my other daemons have propogated the block to my primary daemon yet. It can take quite a while for all of my daemons (let alone external ones) to catch up to the new block.

The cluster I'm running helps here, but clearly this is not sufficient. The electroneum daemon is simply terrible. Sumokoin on the other hand, a "weak" daemon may drop down to 30 connections -- the same number I expect from an electroneum daemon that's working perfectly. A "strong" sumokoin daemon will be able to easily sustain 200 connections.

As all of these are based on open source cryptonote, I have to believe there's some improvements in other daemons, that could be backported to electroneum. Has anyone ever looked at their codebase? Any ideas where to start on this? I would be happy to run an electroneum pool for STH users, but until the daemon issues can be improved, I would be hard pressed to recommend my own pool (or electroneum in general) to STH users. On the other hand, if we can get an electroneum daemon working properly, and run a pool using it, it could provide us a good earnings boost versus other electroneum miners.

Any thoughts or ideas?
 
  • Like
Reactions: yyfong

spfoo

Member
May 23, 2017
102
16
18
I've had even worse problems running an Intense pool. The daemon basically just freezes and stops supplying new blocks to mine. There's no way to know this is happening through the pool, daemon or wallet logs. Everything seems fine. Just if you rescan the wallet it's balance will be 0 when the daemon hangs. The only solution - very sub-optimal - I found was to periodically restart the daemon through cron.

Other coin daemons have been more stable than Intense, but each one that I've tried have had issues. Now all of these cryptonight daemons are based on the same code so I'm suspecting likewise that there is a fundamental flaw that affects every one of them. Maybe the Sumokoin project has a solid network of daemons of their own that helps maintain stability. I haven't checked the source code of the daemon, but I doubt that there are any big changes made in any of the forks. The focus is more on the GUI wallets eg. the stuff that "sells".
 

Jeggs101

Well-Known Member
Dec 29, 2010
1,529
241
63
ETN is 400MH/s XMR is 850MH/s

ETN is 1/10th the value and has no transaction volume.

Maybe the network for ETN is not as good? It has the miners, but maybe the block time design is not as good when you've got lower transaction volume. What % of ETN txns do you think are miners getting paid? 98%? 99%? More?

I don't know. Whenever coins f* with the fundamental block timing catering to miners they run into issues. Maybe it's not just that your not maintaining the daemons that are struggling, maybe its everyone's and it leads to a dysfunctional network of nodes.
 

Joel

Active Member
Jan 30, 2015
850
191
43
42
Honestly, it sounds like the concept of the 1m block time itself is the flaw. It probably results in faster transactions for users, but that's about the only benefit. Stability is clearly suffering...
 

spfoo

Member
May 23, 2017
102
16
18
Honestly, it sounds like the concept of the 1m block time itself is the flaw. It probably results in faster transactions for users, but that's about the only benefit. Stability is clearly suffering...
Good point. The original has a 2 minute block time for a reason maybe. And it's been designed/tested with that in mind.
 

funkywizard

mmm.... bandwidth.
Jan 15, 2017
848
402
63
USA
ioflood.com
Now all of these cryptonight daemons are based on the same code so I'm suspecting likewise that there is a fundamental flaw that affects every one of them. Maybe the Sumokoin project has a solid network of daemons of their own that helps maintain stability.
That's probably part of the story, but there definitely is something specifically wrong with electroneum and not all of the others.

I'm running around a dozen daemons on the same server inside of lxc containers. Two daemons are considered "primary" -- they each connect to all of the other daemons. The 10 "non primary" daemons all connect to the primary two, but not to one another.

The way this is done is "add priority peer". In this configuration, the daemon attempts to stay connected to the priority peer, and reconnect as needed.

You would think in this setup, that the two "primary" daemons would nearly always be connected, at minimum, to the other 11 daemons, plus whatever external connections they manage to complete. (keep in mind they are all located on the same physical machine, with overkill amounts of ram, cpu, and disk i/o). You would also assume, that with all 12 daemons no more than "2 hops" away from one another, they would all learn of each other's blocks in less than 1 second.

Neither of these things occurs. Connections between local daemons are just as flakey as connections to internet daemons. Worst-case, I sometimes see a "primary" daemon with as few as 4 - 5 *total* peers, --including-- the local priority peers it should always be connected to.

If this problem affected all cryptonote daemons equally, you wouldn't see any of them functioning properly. So I believe there is more to this than the parent code being awful. I suspect that either they've forked some very outdated code and never backported any improvements, or they broke a bunch of stuff with their own changes, or both.
 

funkywizard

mmm.... bandwidth.
Jan 15, 2017
848
402
63
USA
ioflood.com
Good point. The original has a 2 minute block time for a reason maybe. And it's been designed/tested with that in mind.
I agree that 2 minutes would be better. It would reduce the size of the problem but unfortunately would not eliminate it. Sumo has, I believe, a 4 minute target time, but this is not quite what it seems. Because the difficulty oscillates so strongly, you'll have a string of 10-20 minute block times, and a string of 30 - 120 second block times, in roughly equal proportion. During the "short block times" phases for Sumo, the network still seems to operate pretty well, whereas with electroneum, even if they had 2x the block time target, it would still be pretty unstable.
 

spfoo

Member
May 23, 2017
102
16
18
That's probably part of the story, but there definitely is something specifically wrong with electroneum and not all of the others.

I'm running around a dozen daemons on the same server inside of lxc containers. .
Yes if that doesn't kill the problem I don't think hardware can solve it. But as you've drilled down pretty deep on the issue I wouldn't think it's hard to fix. Seems like the syncing is not triggered immediately for some reason when a new block is found. It leaves only a few options where the problem could be.

However fixing the connection to the internet nodes is more difficult. getting the local nodes to sync faster is probably fairly straightforward.
 

spfoo

Member
May 23, 2017
102
16
18
I an see the problem very well now. I compiled the daemon a couple of days ago and started a sync. It hasn't finished syncing yet. It stops the sync randomly every now and then.
 

spfoo

Member
May 23, 2017
102
16
18
I think one issue ETN can have is that they have a huge number of miners. Just look at hashparty.io US as an example: 689 miners for just 279 khs. If that's a typical rate it would hint at over a million miners in total. Those numbers put a very different strain on the network.
 

funkywizard

mmm.... bandwidth.
Jan 15, 2017
848
402
63
USA
ioflood.com
I think one issue ETN can have is that they have a huge number of miners. Just look at hashparty.io US as an example: 689 miners for just 279 khs. If that's a typical rate it would hint at over a million miners in total. Those numbers put a very different strain on the network.
Perhaps so, but it depends upon the pools really. A single miner does not necessarily have their own daemon running. The official developers are promoting tbeir centralized web wallet, so few or no daemons there. Then the top 2 pools account for easily half the network hash rate. That could be as few as 2 daemons there. Roughly 3 exchanges trading ETN, so 3 more daemons.

So the bulk of the transactions are probably occuring between less than a dozen nodes.

Obviously there are far more nodes than that, but it's certainly not 1 daemon per 1 miner.
 

spfoo

Member
May 23, 2017
102
16
18
I've been looking into this more through the Intense daemon as ETN has been broken. I've seen similar things that you about the connectivity and lags in getting the update about a new block etc. in that daemon. I just didn't notice it before because it happens only every few hours. But the latest bug I found is the most confusing. This is a block I found yesterday on my pool. From my pools block list:
Height Maturity Difficulty Block Hash Time Found Shares/Diff
139341 586435392 bc0ae069739bef758a839d33393fc9d29a21f2cc81cf52e24a8ca380be18b991 3/5/2018, 9:33:48 PM 44%

Then I looked it up in the block chain explorer:
Height Size Block Hash Time Found Transactions
139342 15695 3318b8c15f40ba2dee581f714e0f7f05c56e691c70bfea5f1fffd9b0f045cddb 3/5/2018, 9:33:48 PM 6
139341 7342 bc0ae069739bef758a839d33393fc9d29a21f2cc81cf52e24a8ca380be18b991 3/5/2018, 9:31:49 PM 2

Notice anything interesting? The time stamps are messed up.
 

funkywizard

mmm.... bandwidth.
Jan 15, 2017
848
402
63
USA
ioflood.com
I've been looking into this more through the Intense daemon as ETN has been broken. I've seen similar things that you about the connectivity and lags in getting the update about a new block etc. in that daemon. I just didn't notice it before because it happens only every few hours. But the latest bug I found is the most confusing. This is a block I found yesterday on my pool. From my pools block list:
Height Maturity Difficulty Block Hash Time Found Shares/Diff
139341 586435392 bc0ae069739bef758a839d33393fc9d29a21f2cc81cf52e24a8ca380be18b991 3/5/2018, 9:33:48 PM 44%

Then I looked it up in the block chain explorer:
Height Size Block Hash Time Found Transactions
139342 15695 3318b8c15f40ba2dee581f714e0f7f05c56e691c70bfea5f1fffd9b0f045cddb 3/5/2018, 9:33:48 PM 6
139341 7342 bc0ae069739bef758a839d33393fc9d29a21f2cc81cf52e24a8ca380be18b991 3/5/2018, 9:31:49 PM 2

Notice anything interesting? The time stamps are messed up.
Interesting. It looks like either the pool or the explorer is off by one.
 

spfoo

Member
May 23, 2017
102
16
18
Turned out as I suspected that the problem is indeed in the volume of transactions caused by large number of small scale mining. The wallet code - copied from Monero - can't handle the transaction volume for long and starts collapsing. That's the root cause of the daemon and network instability as well. Just wondering how little testing has the Electroneum team done on the copied code...