Both are based on the last free OpenSolaris Kernel. While NexentaCore is OpenSource, Solaris Express is only free to use
for non-commercial and demo usage. Otherwise you need a annual subscription from Oracle or you may use OpenIndiana -
the quite similar free alternative to Solaris Express.
Difference beside licensing
Userland: Nexenta is something like Ubuntu-server with Solaris Kernel. Update and installation of tools are done with apt.
Solaris Express and OpenIndiana are pure Solaris with installation of updates and tools are done via the pkg tool.
So you should always compare Nexenta vs SE11/OpenIndiana
NexentaCore is CLI only, while SE11 and OI can be installed optionally with a GUI (Live Version)
Nexenta is based on OpenSolaris Build 134 with a lot of storage related backports from newer builds
Solaris Express (Build 151) is the closed source successor of the last free OpenSolaris Build 148
OpenIndiana (Build 151) is the Open Source Fork of the last free OpenSolaris Build 148,
now based on the Illumos kernel development project
Main difference between Solaris Express and the others
Oracle finished development of encryption with Zpool Version 31,
while all other are at pool version 28 without encryption (the last free version)
Common between Nexenta and OpenIndiana
Nexenta is one of the major contributors behind Illumos, the project that forks the last free OpenSolaris. While current OpenIndiana 151 is already based on Illumos, Nexenta will do with the next Nexenta Version 4.0
Main difference between Nexenta and OpenIndiana
OpenIndiana is intended to be the free and real OpenSolaris successor for general server and desktop use, Nexenta has the focus on Storage (base for the commercial Nexenta Enterprise storage server)