Great news .. Thank U , shall focus on Frenetic / unless MPI has advanced message segmentation at the byte level that supercedes Fr.. its hard to choose from a glut of tools made available .. particularly when one reads a Mellanox Linux OFED install wipes the slate clean to install IB drivers & libraries one has no idea may work with PF_ring or cause instability .. PF_ring has its own installer offering great features (but at what overhead or sacrifice to X3 stability low level tools like Frenetic ?
there is no mention to borrowing from PF_ring (but much from OpenFlow) are these different camp competing ie commercial NTOP
we force the programmer to reason how to compose ''the size of table, space for rules , like register allocation in assembly" thats OK , Frenetic proposes to abstract the switch by API to allow higher level access network functionality .. to process events / direct packets .. what sort of abstraction allows this across many different vendors ? does it scan each switch to provide a resource list specific to each switch , that is the only approach, like running a firmware change must also show that change
it does appear to apply policy to the network , when u try to do that on ConnectX there is going to be a priority contention for some runtime rules, esp statistic reporting or where is 1st packet is directed , is it going to the device controller or ConnectX ASIC
then we have bugs . contentions. caused by callbacks in Frenetic SQL commands.How do other modules affect this( obvious loads up the CPU ) as this is command runtime
I want to know all library dependency and list as kept in cache, how they interact with OFED IB tools and Aopen binaries that FreeBSD is said must continue to interoperate via XFree86,, this level of detail is a must, Aopen and ELF cache must be replaced.
If im going to use Frenetic, i am happy to define specific resources for the X3 card and managed switch , yet Frenetic abstracts this to avail a vendor agnostic program model, i want specific switch features available at the low level ,, it appears Frenetic dodges this by abstraction ..
typically a OFED Linux install begins with OpenSM .. i dont want to run OpenSM with a managed switch , but it says each IB node must have openSM running ,, not very enlightening .. (not asking for walkthru, only better links to running a managed Sw instead of openSM in testing IB on startup (sure this is a small thing , but so many small things (lazy feature obsessed techies) create a nightmare , im not saying the Prof is lazy i shal email her after i compare PF_ring .. on how quickly she apologized for fabrics having limited resources (table size, etc) and the register assignment like approach in dealing with low level .. the promise of modifying packet flow must be at that level or we get an adhoc approach..
Choose a Subnet Manager
Each infiniband network requires a subnet manager. You can choose to run the OFED opensm subnet manager on one of the Linux clients, *
or you may choose to use an embedded subnet manager running on one of the switches in your fabric. Note that not all switches come with a subnet manager; check your switch documentation.
* leaving out how to run a managed switch
Load the kernel modules , what happens when new modules overlap?
Infiniband kernel modules are not loaded automatically. You should adding them to /etc/modules so that they are automatically loaded on machine bootup. You will need to include the hardware specific modules and the protocol modules.
/etc/modules:
# Hardware drivers
# Choose the apropriate modules from
# /lib/modules/<kernel-version>/updates/kernel/drivers/infiniband/hw
#
#mlx4_ib # Mellanox ConnectX cards
#ib_mthca # some mellanox cards
#iw_cxgb3 # Chelsio T3 cards
#iw_nes # NetEffect cards
#
# Protocol modules
# Common modules
rdma_ucm
ib_umad
ib_uverbs
# IP over IB
ib_ipoib
# scsi over IB
ib_srp
# IB SDP protocol
ib_sdp
it says optional ,, but does not provide the alt option , staging openSM ..
(optional) Start opensm
If you are going to use the opensm suetnet manager, edit /etc/default/opensm and add the port GUIDs of the interfaces on which you wish to start opensm.
You can find the port GUIDs of your cards with the ibstat -p command:
# ibstat -p
0x0002c9030002fb05
0x0002c9030002fb06
/etc/default/opensm:
PORTS="0x0002c9030002fb05 0x0002c9030002fb06"
Note if you want to start opensm on all ports you can use the PORTS="ALL" keyword.
Start opensm:
#/etc/init.d/opensm start
If opensm has started correctly you should see SUBNET UP messages in the opensm logfile (/var/log/opensm.<PORTID>.log).
Mar 04 14:56:06 600685 [4580A960] 0x02 -> SUBNET UP
Note that you can start opensm on multiple nodes; one node will be the active subnet manager and the others will put themselves into standby.
How to start it on the managed switch ?
And whats wrong with low-level ? a packet is a packet no matter where until its wrapped in a transport layer .. or trunced by endian reformat, *where does this happen ? why is it not at wirespeed in the adapter ? rather than in a cache binary ? can this be replaced specifically to format in the adapter , its a real nuisance im sure IB / RDMA > RoCE encapsulation is in the X3 ASIC
AS prof states Frenetic has simple abstraction on the data plane " direct control of the underlying switch element rather than baroque mechanisms alternatively offered "
direct control is a minimal demand, what's lacking are atomic logic operators and index handlers on a per context basis .. Jen is a proficient speaker but demonstrates a black box rule based network traffic inspection modular environ ,, showing the issue of the packet reaching the switch before the rules (the example trnsitions between policy 1 or 2 and not bits of each .. the per packet update is curious, i guess i leave some questions on the utube clip.. Many questions unanswered.