Supermicro Xeon-D 1587 Squid Performance

Notice: Page may contain affiliate links for which we may earn a small commission through services like Amazon Affiliates or Skimlinks.

tssrshot

Member
Mar 18, 2015
58
8
8
Omaha, NE
It's almost as though we've come full circle back to "bug the source of the download to come up with a binary delta format". I understand that the current format is stupid--that's what should get fixed. :) If there's only 50k of changes, you should only be downloading 50k, not 30G. This isn't exactly a new problem, and instead of trying to come up with crazy workarounds like caching servers the vendor should just provide something that makes more sense.
I can appreciate how silly it sounds. I assure you only a small part of it is a technical issue. In fact, really the biggest issue is administrative, but causes a technical issue. You're right in that airports don't change often, but their "environment" does, which can include up to 250 nautical miles around it. Marry that with towers, poles, cranes, wind farms, etc...and you've got a chart section that is changing every day. You can possibly lose your mind about it, even the USDA will review some of these areas to determine the average growth rate of the trees near certain fields and forecast how high they will be and call it an "obstacle." You're average Delta jet isn't as concerned, but Visual Flight Rules aircraft like a Cessna or Mooney can't make the climb over a new wind farm without some notice and study.

So yes, some of it can be fixed with better design, but I think this company and many other agencies are "wait and see" on digital formats. There is some progress in that some of the charts are moving to mbtile format which will move a chunk to ~500KB small tiles. Which ought to allow for more delta opportunity, but possibly overwhelm older devices with the number of small changes it has to process. It's a double-edged sword.

I wouldn't be pursuing such an unorthodox solution, but as they say "Time is Money." We have on average 500 more Pilots and 1500 more Aircrew members in small space than a normal airport, because we are a large scale training environment. Students without valid data in hand are not only FAA non-Compliant, but possibly about to embark on a dangerous flight. I'm lucky in that our other projects fund ridiculously overkill items I can use to solve unique problems. I'm actually over my head/underwater on a method to use SAML with Canvas LMS + Software issued certificates...so don't think this is all I'm thinking about. :)

Since everyone is eyes on, any chance someone can tell me if this dynamic squid config is valid?

Code:
refresh_pattern ([^.]+\.)simperium.com/.*\.(zip|sqlite|sl|pdf|png|jpg|gif|fbtiles) 4320 100% 86400 reload-into-ims;
refresh_pattern ([^.]+\.)segment.io/.*\.(zip|sqlite|sl|pdf|png|jpg|gif|fbtiles) 4320 100% 86400 reload-into-ims;
refresh_pattern ([^.]+\.)foreflight.com/.*\.(zip|sqlite|sl|pdf|png|jpg|gif|fbtiles) 4320 100% 86400 reload-into-ims;
 
Last edited:

tssrshot

Member
Mar 18, 2015
58
8
8
Omaha, NE
Anyone know off the top of your head why the first statement works but the not the following three?

refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims;

refresh_pattern ([^.]+\.)foreflight.com/.*\.(zip|sqlite|sl|pdf|png|jpg|gif|fbtiles) 4320 100% 86400 reload-into-ims;

refresh_pattern ([^.]+\.)simperium.com/.*\.(zip|sqlite|sl|pdf|png|jpg|gif|fbtiles) 4320 100% 86400 reload-into-ims;

refresh_pattern ([^.]+\.)segment.io/.*\.(zip|sqlite|sl|pdf|png|jpg|gif|fbtiles) 4320 100% 86400 reload-into-ims;
 

amalurk

Active Member
Dec 16, 2016
311
116
43
102
Anyone know off the top of your head why the first statement works but the not the following three?

refresh_pattern ([^.]+.|)(download|adcdownload).(apple.|)com/.*\.(pkg|dmg) 4320 100% 43200 reload-into-ims;

refresh_pattern ([^.]+\.)foreflight.com/.*\.(zip|sqlite|sl|pdf|png|jpg|gif|fbtiles) 4320 100% 86400 reload-into-ims;

refresh_pattern ([^.]+\.)simperium.com/.*\.(zip|sqlite|sl|pdf|png|jpg|gif|fbtiles) 4320 100% 86400 reload-into-ims;

refresh_pattern ([^.]+\.)segment.io/.*\.(zip|sqlite|sl|pdf|png|jpg|gif|fbtiles) 4320 100% 86400 reload-into-ims;
Never used squid and I suck at regex but all (even first) look like they have an un-escaped / before the domain suffix unless you aren't required to escape that here.
 
  • Like
Reactions: tssrshot

tssrshot

Member
Mar 18, 2015
58
8
8
Omaha, NE
Sorry for the long pause, Gang. I appreciate all the inputs. Turns out that the vendor was mix/matching https and http which is why the Squid config wasn't doing its job. Now that we employed some HTTPS intercept, things are much improved.

I had to find a few "forests for the trees" to get things flowing, but even after everything the SuperMicro 1587 is a beast. If anyone finds this in curiosity, feel free to message with any questions or suggestions.