Looking for VPS or dedicated Server with near perfect latency to Phoenix (Arizona)

Hello there,

I hope the forum section is about right.

I am trying to find a vps / or dedicated server that has a very low latency to a server that I think is located in Phoenix, Arizona...
ATM I use a Vultr VPS 1 Core 1GB Ram. Ubuntu.

The ping from this vps to the server api.poloniex.com is 0.500 ms.
Unfortunatly for my use case this is either not fast enough, or vultr steals too much of the cpu time...

Does anyone know a good (and not too expansive) hoster?

If you have access to a vps / dedicated server that you think is good I would be really really really gratefull if you could post the results of

curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal:%{time_total}\n' 'https://poloniex.com/public?command=returnOrderBook&currencyPair=USDT_BTC&depth=10'

My results are
lookup: 0.004203
connect: 0.004762
appconnect: 0.016801
pretransfer: 0.016902
redirect: 0.000000
starttransfer: 0.078210
total:0.078307

 

So you want to high frequency trade on Poloniex and their server is in a Phoenix DC? Figure out which DC and then rent 1U in the same datacenter.....

 

Considering api.poloniex.com is on cloudflare's CDN the rate at which they push content through cloudflare may be your limiting factor and you won't get much better unless you can circumvent the CDN by being in the same data center closest to where they push the updates to the CDN from, or getting a direct connection to them.

For example I have sites in Canada and the UK that are faster than the timings above and only 3 hops @ 0.162ms ping and 4 hops @ 0.4ms but connections from Phoenix, Boston, NY, LA @ 1 Wiltshire etc are slower. The appconnect and pretransfer times are going to vary more as well based on the CPU in question as a good chunk of that is SSL/Crypto stuff and warming up the caches on the CDN side.

 

@Blinky 42 sorry I am not a native english speaker: Do I understand you correct, that you habe a site in Canada (or the UK) that has a 0.162ms ping to api.poloniex.com? Is that a ups / dedicated hosting company?

Concerning the app connect and pretransfer timings: I have tried with a vps with 2vcpu but nothing changed there

 

Correct - my site in Toronto is in a large Telco hotel downtown, and Cloudflare has a set of Content Distribution Servers in the same building that is 3 hops away (my router, the intermediate router and then Cloudflare).

Because Cloudflare is a CDN - they have servers all over the planet peered (directly connected) to many ISPs and last mile providers in each city.
Our Anycast Network Map | Cloudflare

So when you go to api.poloniex.com it should actually make a connection to the closest server in terms of network distance. Clients of cloudflare like Poloniex make their content available on their private servers and use Cloudflare as the front end distributed proxy around the planet. That way they can focus on building their software and just pay Cloudflare to be able to handle any level of traffic that may hit the API from around the world, without spending millions on dedicated servers in hundreds of datacenters around the world and managing them 24x7.

If you want an accurate latency measurement then you need to check against a API that has time stamped data within and see what the delta between the transaction time and wall tie is when you can see that data. HTTP(s) api's like this are far from optimal for real-time data where overall latency is important. At a minimum you need to use an interface that allows a single connection the be maintained between you and the server and have multiple transactions pushed from the server to you as they happen. One of the main use cases of real-time messaging platforms (like IBM's Websphere MQ, and more recently AMQP servers like RabbitMQ etc) was for high-volume low latency distribution of trading data.

 

A more detailed look at where cloudflare peers can be found here PeeringDB

 

At the end of the day the public API is not intended to handle this I assume. The trading company’s usually charge a large chunk of cash to use a private interface and will you what DC to use to connect.

 

@Blinky 42 Thank you very much for the further explanation... I think I understood the Cloudflare part... Still that makes it kind of difficult to choose a location and then do the neccessary operations to evaluate if the location is performing better than the one I already use...
I wont pretend that I understood the part with the accurate latency measurement, but I think all hope is lost there, because I am too much of a newbie concerning those things. The theory is understood, but I dont have a single clue how to get this done... But thanks anyways... I think I will do some trial and error

 

Aside from the technical how to do it - @Evan got probably the most important point: this API probably isn't designed for high frequency trading or that level of fast access. You should check and see what they allow you to do with the API before spending a lot of time and effort to get a super-fast connection to something that isn't updated that often. Just casual looking at the main site, updates seem to be at about 4Hz.

Good luck in your project, but do be careful they don't tag you as abusing their network. You may be better off contacting them to get access to a higher bandwidth direct feed of trade details.

 

I will do that. Tank you very much guys.

 

@Blinky 42 and all others that would like to chime in.

May I kindly approach you with a few more questions?

1.) Just to clarify as my english is not that good:
so, the goal is really to find a quick connection to cloudflare's CDN, as content is served from there, and the connection to poloniex is actually irrelevant? In that case, i'm guessing it's best to find a decent, affordable dedicated server in the same building as a cloudflare CDN location (or near it)? is it important to find one near an IXP such as coresite?


2.) also i've been looking for dedicated servers.
This is a rather small operation, not sure that renting server based on a E3 1245 V3 / 8GB DDR3 / Intel NIC, is worth the trouble. Would a dedicated server solve some of the local latency issues (stolen time, etc), or is it worth it to get something more involved? I won't have physical access to the site, so whatever I get it needs to be setup and maintained by them.
With most of the bigger operations (vultr, DO, Linode) it doesn't seem to matter comparing the results if I chose a dedi or a vps with just 1 core and 1gb ram...

3.) Would tweaking the network stack be of any help to get the latency down?

4.) Would a real-time Kernel or a low latency kernel do any good?

@Patrick: If my memory serves me right: Don't you have some machines hosted in the bay area?
Just out of curiosity: Would you have the time to run a
curl -L --output /dev/null --silent --show-error --write-out 'lookup: %{time_namelookup}\nconnect: %{time_connect}\nappconnect: %{time_appconnect}\npretransfer: %{time_pretransfer}\nredirect: %{time_redirect}\nstarttransfer: %{time_starttransfer}\ntotal:%{time_total}\n' 'https://poloniex.com/public?command=returnOrderBook&currencyPair=USDT_BTC&depth=10'

command on one of the machines (with a low latency?)


Hopefully that is not asking too much of your time...

Thank you (and others that may chime in) for your time and help

 

Okay, without being overly repetitive, let’s just say that you might be looking for a solution to the wrong problem. As @blinky42 said it best, the Poloniex API box is sitting behind a Cloudflare CDN (content delivery network) box. All a CDN does is ensure availability by allowing clients to connect to their local CDN node, and then forward the requests over to the actual server hosting the trading market - it doesn't guarantee fastest market access. Think of it this way - Cloudflare has a CDN node in geographies such as Phoenix, SFBA, Toronto and who knows. The truth of the matter is that we are not sure where Poloniex hosts their market server. I am also not sure if the same API box is also responsible for both order execution and sending out market data. There are processing overhead dealing with the CDN node, and network propagation time for the trade signals to move from the CDN to the actual market hosting machine (the venue). If you are using a NY based CDN and the venue machine is in Montreal, great. If it’s located in Boulder, Colorado, that’s less good. I am pretty sure Poloniex does not do any CDN mode response time qualization to ensure that a CDN node farther away gets the same response time as the CDN nodes closer by. In order for you to build a formula 1 race car to beat the others you’ll have to know that there are no speed bumps and hills ahead of you - with an even playing field do you engage your bag of tech tricks.

Let’s take a look at how low latency is typically used, based on a previous gig of mine at an established electronic trading venue (algo based prop trading) - Let’s just say that you trade commodities with the CME (Chicago Merchantile Exchange). There is a CME owned data center near Aurora, Illinois (about 30 minutes drive from Downtown Chicago) that is owned by the trade venue - the trade venue guarantees that all market participants in the facility are wired equidistant to the venue’s computers. You pay for colocation space with them, register as a trader, pay market access fees, and then pick from a myriad of market data delivery options from various markets, either Microwave, fiber, RDMA, etc so your algos can gather info faster than others and have the ability to fire trade commands out the quickest. In that case, you’ll look at options like kernel bypass (SolarFlare OnLoad), CPU affinity setting/dedicated Interrupt handling (irqbalance is a dirty word here), deliberate shut off of cores to allow higher sustained turboboost, polling instead of interrupt based operations to get stuff in and out of the wire. Just remember that speed is life, and speed will cost you money. Also, it's fairly important to keep tabs on system performance and know where to invest time and money where it counts.

 

Thank you so much

 

Thank you for your posting... Sorry if I am a bit slow in understanding... Still improving my language skills.

When it comes to crypto trading there seems to be no datacenter like that.
There is one provider in NY that is ridiculous expensive (I have just 1k USD to play with and the cost 1700 a month) the have a ping of 0.7ms to poloniex (they say - offering no trials).... and I heard you shouldn't use NY, because everything that is crypto relatet is forbidden there.

So keeping in mind that I am the average guy that spent 250 bucks on a arbitrage tradingbot and is trying to make this thing pay off somehow to get an ROI I was aking my favorite computerfolks here if the could assist me with some ideas on where to set up a dedi or a vps...

My bot has the right settings, but latency is in his way to actually do the trades before they are grabed away by someone else...
And of course: nobody of the big earners wants to share there hosters, becauese every guy that is the fastest racer will get the fishes (trades) out of the same pond (exchange).

Well so it is for me to do more trial and error then...

 

Well, here's the thing, and this is from a guy who worked in the prop-trading/HFT business for 3 years...any trading opportunity based on strategic arbitrage (stratarb) will not last long enough for you to capitalize majorly upon it, especially not in such a distorted market like crypto-currency (low liquidity, tons of iceberg traders, possibilities of market manipulation, etc). If I have to guess a strategy here, it'll be to keep a few AWS micro instances (or its google cloud/azure/digital ocean equivalents) up in several geographies, "ping" the API box on a recurrent basis to look for patterns over a time-series, and then use that data to see if you can trade to your advantage. If you know that, say, the CDN in Montreal has a lower overall ping value than the one in Kansas City every Tuesday, maybe you can write some code to process orders there, and etc. it might not be this simple, but then, no one said moneymaking is easy.

 

As @WANg said, it may be tough to make back your investment in this market segment - but if it was easy everyone would be doing it

The goal is NOT to find a fast connection TO cloudflare - if you are trying to do trades before others in the same market are doing, you really want to go around cloudflare and directly to poloniex somehow. With your budget mentioned above, I suspect you will not be able to achieve that unless you know people at the company who can give you more information on a connection that is not through cloudflare.
Assuming that you must use the cloudflare-based API then you would need to find something in the data to/from poloniex that you can use as a time stamp in real-time (like a transaction time with millisecond resolution) to be able to compare times of trades vs. how long it takes to see those trades at the site in question. Barring that, you just need to experiment with a mix of sites and see what you get performance wise.

From my Toronto site a moment ago (on a not super zippy cpu so ssl stuff is slower):

lookup: 0.004
connect: 0.005
appconnect: 0.133
pretransfer: 0.133
redirect: 0.000
starttransfer: 0.161
total:0.161
[root@yyz-vm-01 ~]# traceroute poloniex.com
traceroute to poloniex.com (104.20.12.48), 30 hops max, 60 byte packets
1 xx.yy.zz.aa (xx.yy.zz.aa) 0.309 ms 0.302 ms 0.304 ms (my router)
2 cloudflare-ic-309911-toro-b1.c.telia.net (62.115.52.138) 6.276 ms 6.272 ms 6.325 ms
3 104.20.13.48 (104.20.13.48) 0.205 ms 0.186 ms 0.166 ms

[root@yyz-vm-01 ~]# ping -nqc 100 poloniex.com
PING poloniex.com (104.20.12.48) 56(84) bytes of data.

--- poloniex.com ping statistics ---
100 packets transmitted, 100 received, 0% packet loss, time 99001ms
rtt min/avg/max/mdev = 0.162/0.189/0.308/0.025 ms

 

Well, direct colocation is obviously the way to go if you are planning to do arbs. But then, if you do any prop trading (algo/quant-driven), you will want to colocate just for a place on the big trade tables. As for getting behind the Cloudflare and talking directly to the venue? Eh, that's probably a no-go in most cryptocurrency exchanges. At the very least, you are looking for an informational advantage that venues will not let go all that easily. They also don't want to reveal too much and invite attacks. The CDN is an important layer in their defense.

Like I said, if your strategy is to take advantage of timing to do arbs, you'll need information on timing. You'll need to ping them from various locations and see if there is a pattern. Not every CDN will behave uniformly - if they are smart they'll put in some kind of minimum latency thing or some randomization just to throw people off-guard. Look at their API, gather stats from different locations, and if there is a market venue timestamp, see if there are consistent patterns. How would you know what "perfect latency" is if you don't have any idea what their average, mean, median, spread and best/worst latencies looks like, and whether geography plays a role or not?

And oh yeah? Remember that old joke about how to become a millionaire playing the market? Start with being a billionaire.

 

Thank you again @Blinky 42 and @WANg for the great insights you provide. I am happy to learn and really appreciate your input and our conversation very much.

There was a comment here about http/s and how it has issues with responsiveness.
According to the official docs, all poloniex API calls use wss (websocket).
Is this faster? Or better? Or easier to write scripts for?

About measuring time..
i've received adivce elsewhere to write a ruby script.
The built-in benchmarking looks pretty easy to use.
Would this be a good way to proceed? And would ruby on rails be required to send requests to the server?

And about wrapping the whole thing up into a script that actually runs.. the buy command is listed in the poloniex docs, and has an immediateOrCancel option. So i can see all the pieces, but for a non-programmer syntax will be a challenge. Are there any good beginner tutorials that can help me to write this thing?