CD-EXCALIBUR-FMS0067-M6c.1_ExcaliburFusionModellingSystem
=========================================================

.. meta::
   :description: technical note

   :keywords: ExCALIBUR,Management,of,external,research.,Spectral,El-,ements,and,Referent,Model,Procurement,M6c.1,Abstract,The,report,describes,work,for,ExCALIBUR,project,NEPTUNE,at,Mile-,stone,M6c.1.,It,collates,technical,material,used,to,inform,the,preparation,of,Calls,Contract,T/AW085/22,and,Contract,T/AW086/22.,The,material,is,better,presented,with,that,relating,the,latter,contract,appearing,ﬁrst,,so,the,report,can,examine,the,challenges,for,computational,physics,before,the,speciﬁcs,of,spectral,ﬁnite,elements.,Regarding,T/AW086/22,(“Advanced,referent,model,procurement”),,there,is,a,refresher,of,the,time,and,spatial,scales,involved,in,(i),classical,ﬂuid,dynamics,,(ii),neutral,species,from,the,kinetic,standpoint,,and,(iii),tokamak,plasma.,Drawbacks,of,three,different,variational,approaches,to,ﬂuid,dynamics,are,also,described.,Regarding,T/AW085/22,(“Spectral,element,procurement”),,in,order,to,clarify,the,scope,of,future,work,,attention,is,given,to,accurate,description,of,what,is,currently,available,within,the,code,,speciﬁcally,(i),the,current,status,with,respect,to,usage,on,HPC,,(ii),meshing,,(iii),ﬁnite,element,capability,with,respect,to,possible,FEEC,usage.,A,modal,discontinuous,Galerkin,approach,to,elec-,tromagnetics,is,described,in,an,appendix.,UKAEA,REFERENCE,AND,APPROVAL,SHEET,Client,Reference:,UKAEA,Reference:,CD/EXCALIBUR-FMS/0067,Issue:,Date:,1.00,October,9,,2022,Project,Name:,ExCALIBUR,Fusion,Modelling,System,Prepared,By:,Name,and,Department,Wayne,Arter,Ed,Threlfall,Joseph,Parker,Will,Saunders,Signature,N/A,N/A,N/A,N/A,Date,October,9,,2022,October,9,,2022,October,9,,2022,October,9,,2022,BD,Reviewed,By:,Wayne,Arter,October,9,,2022,Project,Technical,Lead,1,1,Introduction,This,report,provides,a,literature,review,designed,to,inform,the,production,of,Calls,T/AW085/22,“Spectral,element,procurement”,and,T/AW086/22,“Advanced,referent,model,procurement”.,Ide-,ally,the,physical,model,of,the,plasma,should,be,derived,with,a,view,to,efﬁcient,numerical,imple-,mentation,in,general,and,at,Exascale,in,particular.,Such,treatment,is,a,key,part,of,the,role,of,the,computational,physicist,,see,Section,2.,In,practice,,there,is,a,need,to,demonstrate,as,rapidly,as,possible,that,the,drift,kinetic,model,proposed,by,Parra,et,al,[1],is,feasible.,This,has,meant,use,of,a,numerical,approach,with,which,the,Oxford,group,is,comfortable,,namely,a,Chebyshev,spec-,tral,ﬁnite,element,implementation,in,Julia,rather,than,the,spectral/hp,elements,used,in,Nektar++,,where,developments,are,proceeding,almost,independently.,Generally,the,Parra,et,al,model,with,Oxford’s,numerics,have,performed,successfully,,to,the,extent,that,the,Advanced,referent,model,procurement,needs,comparatively,little,direct,input.,However,,a,hopefully,minor,numerical,difﬁculty,has,been,encountered,,see,Section,2.2,The,report,continues,by,highlighting,typical,challenges,presented,by,the,key,physics,to,be,modelled,,starting,with,an,appreciation,of,neutral,particles,in,Section,3.1,,then,moving,onto,issues,raised,by,the,need,to,model,tokamak,plasma,in,Section,3.2,from,plasma,core,to,wall,,notably,the,sizes,of,the,electric,ﬁeld.,Section,3.3,examines,whether,the,model,equations,might,be,better,expressed,as,a,vari-,ational,principle,,but,concludes,that,this,unlikely,to,be,helpful.,These,latter,two,sections,play,a,largely,conﬁrmatory,role,for,Parra,et,al’s,approach.,Section,4,discusses,material,relevant,to,the,spectral,element,procurement.,The,main,considera-,tion,is,the,production,of,a,2d3v,proxyapp,,involving,libraries,of,spectral,elements,including,meshing,capabilities.,The,work,is,summarized,in,Section,5.,2,2,Challenges,for,Computational,Physics,2.1,General,Knowledge,of,the,smallest,length,and,timescales,that,could,be,important,in,a,model,is,in,general,critical,for,a,reliable,determination,of,its,properties.,Typically,the,model,is,a,nonlinear,partial,differential,equation,and,the,properties,that,are,sought,,are,its,solutions.,It,is,known,,see,eg.,ref,[2,,3],that,failure,to,represent,the,smallest,scales,adequately,can,lead,to,catastrophic,code,failure,,and,that,even,when,for,example,numerical,dissipation,is,added,to,suppress,these,failures,,properties,such,as,wave,propagation,speed,will,be,subject,to,large,errors,unless,the,wavelength,is,adequately,represented,by,the,discretisation.,Almost,regardless,of,whether,machine,is,Exascale,or,not,,it,is,helpful,to,be,able,to,work,with,the,most,economical,discrete,representation,of,a,ﬁeld,,which,might,simply,be,a,function,of,position,or,depend,on,velocity-space,coordinates,also.,It,will,be,assumed,that,already,for,example,a,solenoidal,2-D,vector,ﬁeld,has,been,reduced,to,a,scalar,representation,in,terms,of,the,ﬂux,as,say,a,function,of,Cartesian,coordinates,f2D(x,,y).,A,discrete,representation,implies,point,samples,,in,the,simplest,case,merely,values,of,a,scalar,ﬁeld,f,at,different,points,in,space.,To,model,interaction,with,other,ﬁelds,or,calculate,derivatives,of,f,,,it,is,necessary,to,interpolate,to,evaluate,f,at,other,locations.,Interpolation,requires,knowledge,about,the,likely,behaviour,of,f,at,other,points.,The,information,available,is,that,f,relates,to,a,solution,of,an,advection-diffusion,problem,in,n,space,dimensions,and,time,,where,n,is,up,to,6,as,Vlasov,and,Fokker-Planck,type,equations,are,allowed.,Sources,are,an,additional,complication.,Considering,the,limit,where,diffusion,vanishes,,it,is,evident,that,it,is,not,always,adequate,to,assume,that,f,is,continuous,,since,information,about,f,propagates,along,streamlines,,eg.,f,(x,,y,,t),=,f,(x−vt,,y,,0),in,the,simple,case,of,a,1-D,ﬂow,v,in,the,x-direction.,In,the,absence,of,viscosity,there,is,perfect,memory,of,initial,conditions,,so,the,discontinuities,propagate,unmodiﬁed.,Fortunately,even,a,small,number,of,collisions,eventually,smoothes,away,discontinuities,,however,it,should,already,be,apparent,that,different,interpolation,strategies,will,be,optimal,in,different,locations,depending,on,initial,conditions.,Classical,ﬂuid,dynamical,problems,are,of,two,sorts,,loosely,speaking,depending,on,whether,they,are,produced,by,mechanical,action,such,as,pumping,and,aerofoil,motion,,or,by,body,forces,such,as,buoyancy.,The,former,,properly,characterised,by,the,generation,of,ﬂuid,vorticity,at,material,boundaries,are,of,limited,application,to,NEPTUNE.,In,both,sorts,,but,particularly,the,latter,,there,In,this,phase,,the,optimal,is,a,ﬂow,start-up,phase,in,which,a,linearised,treatment,is,insightful.,representation,will,often,consist,of,just,one,or,two,coefﬁcients,of,the,most,unstable,eigenmodes,which,may,have,a,simple,trigonometric,function,dependence,on,one,or,more,spatial,coordinates,,eg.,refs,[4,,5].,As,solutions,tend,to,(quasi-)steady-state,,particularly,when,the,viscosity,is,relatively,small,,gradients,tend,once,more,to,steepen,due,now,to,non-linear,effects.,Converging,advective,ﬂows,frequently,form,shocks,,although,the,physics,suggests,that,such,discontinuities,will,be,iso-,lated,from,one,another.,The,challenge,in,ﬂuid,dynamical,modelling,is,not,therefore,merely,to,produce,optimal,approxima-,tions,,but,to,maintain,optimality,as,the,ﬂow,structure,evolves,in,some,typically,nonlinear,way.,This,is,one,of,the,key,challenges,addressed,by,computational,ﬂuid,dynamics,CFD.,Use,of,high,order,approximations,adds,to,the,complexity,,see,the,books,by,Boyd,[6],and,more,comprehensively,3,Karniadakis,and,Sherwin,[7],who,describe,the,spectral/hp,element,method,whereby,ﬁnite,element,size,h,and,local,order,of,polynomial,representation,p,are,allowed,to,vary,in,pursuit,of,optimality.,2.2,Numerical,Problems,in,Work,of,T/NA085/20,Problems,of,grid-scale,oscillation,have,been,encountered,when,outﬂow,boundary,conditions,rather,than,periodic,conditions,are,imposed,on,solutions,of,the,kinetic,equations.,The,appearance,of,the,problem,may,be,related,to,use,of,a,coordinate,system,in,velocity,space,which,is,in,translation,at,the,drift,speed,relative,to,laboratory,coordinates.,Thus,at,the,wall,the,boundary,,ie.,where,the,density,function,has,to,vanish,,is,(in,the,present,formulation),not,ﬁxed.,This,leads,to,a,situation,where,there,is,a,physically,very,rapidly,variation,of,density,potentially,causing,difﬁculties,as,above.,However,,there,are,other,possibilities,,although,the,above,seems,best,to,explain,why,the,issue,arises,in,certain,cases,and,not,others.,3,Physics,Challenges,3.1,Neutrals,For,the,case,of,neutral,particle,modelling,,estimates,of,scales,based,on,the,mean,free,path,(mfp),λ,and,collision,time,τ,(or,alternatively,average,speed,(cid:104)v(cid:105)),are,frequently,adequate.,Indeed,,taking,mesh-size,h,=,O(λ),may,well,be,excessively,conservative.,To,see,this,,note,that,the,viscosity,D,(and,indeed,any,transport,coefﬁcient,expressed,as,a,diffu-,sivity),is,of,order,λ(cid:104)v(cid:105),[8,,§,9],so,that,the,mesh,Reynolds,(Peclet),number,is,Since,temperature,T,by,deﬁnition,satisﬁes,1,the,sound,speed,,thus,2,m(cid:104)v2(cid:105),=,3,2,kT,,,(cid:104)v(cid:105),may,be,estimated,as,O(,Reh,=,U,h,D,=,U,(cid:104)v(cid:105),h,λ,Reh,=,MA,h,λ,√,(1),kT,),,ie.,(2),where,MA,is,the,Mach,number,and,so,provided,the,ﬂuid,model,is,adequate,,it,is,only,necessary,that,h,<,λ/MA,(3),Further,,for,neutrals,,λ,may,be,estimated,in,terms,of,the,diameter,of,the,sphere,of,inﬂuence,d,as,λ,=,√,1,2πN,d2,(4),where,N,is,the,number,density,and,a,Maxwellian,distribution,of,particle,speeds,is,assumed.,The,diameter,d,is,deﬁned,as,the,average,of,the,diameters,of,the,particles,involved,in,the,collision.,4,3.2,Plasma,in,a,tokamak,Unfortunately,estimating,smallest,length,and,timescales,when,charged,particles,are,present,is,more,involved.,For,the,case,of,a,fully,ionised,plasma,of,a,single,particle,species,,textbooks,give,three,timescales,for,equilibration,,for,ions,to,thermalise,,electrons,to,thermalise,,and,the,longest,timescale,is,for,ions,and,electrons,to,reach,the,same,temperature.,The,smallest,lengthscale,that,should,be,considered,is,the,Debye,length.,These,points,are,described,very,nicely,in,the,early,chapters,of,the,textbook,by,Helander,and,Sigmar,[9].,From,the,numerical,point-of-view,,models,to,treat,species,which,are,far,from,thermal,equilibrium,must,account,for,the,non-Maxwellian,distribution,of,particle,velocity,,either,by,use,of,(super)-,particle,sampling,or,by,introducing,extra,dimensions,to,represent,the,variation,of,particle,density,in,velocity,as,well,as,position,space.,Plasma,species,closer,to,Maxwellian,may,be,treated,more,like,classical,ﬂuids,but,with,additional,terms,to,represent,deviations,from,the,normal,distribution,which,have,been,averaged,over,velocity,space.,CFD,techniques,for,classical,ﬂuids,may,be,considered,in,these,circumstances.,The,greater,duration,of,the,time,taken,for,the,ions,and,electron,to,achieve,a,common,temperature,means,that,it,is,frequently,appropriate,to,treat,each,species,as,a,separate,ﬂuid,with,a,different,mean,velocity,and,temperature.,Many,of,the,effects,due,to,inter-species,interactions,produce,terms,resembling,viscosity,and,thermal,conduction,,so,that,classical,CFD,techniques,become,even,more,appropriate,,as,only,the,energy,equipartition,term,is,without,some,kind,of,classical,counterpart,,and,to,a,ﬁrst,approximation,,these,plasmas,obey,a,pointwise,Ohm’s,Law,,ie.,current,density,proportional,to,electric,ﬁeld,expressed,as,J,=,σEE,,where,from,[10],,assuming,Te,is,measured,in,eV,,,then,σE,=,0.0239,·,m2s−1,(5),(Te)3/2,ZΛ,However,,the,two-ﬂuid,equations,are,stiff,in,several,senses,[11].,Not,only,is,it,the,case,that,the,momentum,equations,for,ue,and,ui,have,timescales,in,the,ratio,mi/me,≈,1836,even,for,Hydro-,gen,ions,,but,also,the,electrostatic,forces,are,relatively,enormous.,The,latter,may,be,seen,from,Coulomb’s,Law,if,the,electrostatic,potential,is,introduced,so,that,E,=,−∇φ,,then,the,ratio,ρc,(cid:15)o∇,·,E,=,s,qsns,(cid:80),L2,n,(cid:15)o∆φo,(6),where,∆φo,is,a,typical,ﬂuctuation,level,in,the,potential,,assumed,to,have,a,length-scale,Ln,com-,parable,to,the,width,of,the,SOL.,Writing,(cid:80),s,qsns,=,eno∆n,,Equation,(6),can,be,reexpressed,as,ρc,(cid:15)o∇,·,E,=,noe2,(cid:15)ome,L2,n∆n,(e∆φo/me),=,ω2,pe,L2,n∆n,(,1,2,v2,o),.,(7),The,quantity,ωpe,is,the,plasma,frequency,,which,for,a,reference,density,no,=,1020m−3,is,5.6,×,1011s−1.,If,the,speed,vo,is,identiﬁed,with,the,electron,sound,speed,vthe,then,ωpe/vthe,=,1/λD,,where,λD,is,known,as,the,Debye,length,,and,since,clearly,λD,<<,1,,it,follows,since,all,three,ratios,in,Equation,(7),must,be,unity,,that,tokamak,plasmas,are,forced,by,the,electric,ﬁeld,to,be,quasi-neutral,,meaning,∆n,<<,1.,The,consistent,treatment,of,the,electric,ﬁeld,is,a,major,issue,for,models,which,seek,to,model,signiﬁcant,regions,of,the,core,plasma.,This,is,most,graphically,seen,when,the,Braginskii,electrical,5,conductivity,is,evaluated,for,parameters,representative,of,a,reactor,plasma,core,,viz.,Te,=,25,keV,,n,=,1020,m3,,eg.,by,using,the,SMARDDA-MISC,software,[12],,whence,an,estimate,σE,=,5,×,109,Ω−1m−1.,Assuming,currents,of,order,several,M,A,in,a,cross-sectional,area,of,order,square,metres,,ie.,current,densities,of,106,MAm−2,,implies,an,electric,ﬁeld,of,order,10−3,Vm−1.,Thanks,to,the,strong,temperature,dependence,in,Equation,(5),,the,electrical,conductivity,is,much,lower,when,the,plasma,temperature,falls,towards,the,values,of,10,eV,found,in,the,edge,regions,,so,the,central,electric,ﬁeld,is,tiny,compared,to,the,experimentally,measured,electric,ﬁelds,in,the,SOL,of,order,10+3,Vm−1.,3.3,Variational,principles,Finite,element,equations,are,easily,expressed,if,there,is,a,variational,principle,governing,the,sys-,tem,dynamics.,For,non-dissipative,discrete,physical,systems,,this,is,usually,the,case,,so,that,there,are,variational,principles,available,for,the,corresponding,continuum,models,such,as,the,Vlasov,equation,and,inviscid,ﬂuid,dynamics.,In,the,continuum,case,,where,there,is,bulk,ﬂow,,there,are,a,number,of,possibilities,,Larsson,[13,,§,1],recognises,three,,namely,1.,A,straightforward,generalisation,of,the,Lagrangian,from,the,discrete,system,2.,A,‘constrained’,Lagrangian,approach,,whereby,mass,and,energy,conservation,are,imposed,as,constraints,on,the,variation.,3.,A,Lagrangian,approach,whereby,only,mass,and,energy,variations,consistent,with,the,ﬂow,perturbations,are,allowed.,Possibility,1,leads,to,a,variation,conducted,in,coordinates,moving,with,the,ﬂow,(Lagrangian,co-,ordinates),,rather,than,the,more,convenient,coordinates,ﬁxed,in,the,laboratory,frame,(Eulerian,coordinates).,Introducing,Eulerian,coordinates,via,the,constraints,,Possibility,2,gives,rise,to,the,‘Lin’,variables,which,many,ﬁnd,somewhat,mysterious,,but,in,any,event,their,introduction,doubles,the,number,of,variables,in,the,system,to,be,solved,for,,ruling,out,the,approach,on,grounds,of,cost.,Possibility,3,is,typically,approached,using,the,machinery,of,Lie,derivatives,which,is,found,exces-,sively,mathematical,by,many,,thereby,putting,them,off.,However,,P3,appears,less,formidable,when,it,is,understood,that,this,machinery,was,introduced,by,Lie,precisely,to,deal,with,ﬂow,problems,,and,in,any,event,as,well,as,Larsson,[13],,there,are,textbooks,[14,,15],which,provide,translation,to,more,familiar,vector,notation.,Simply,stated,and,unsurprisingly,in,the,light,of,their,manner,of,introduction,,allowed,variations,are,constrained,to,take,the,form,of,Lie,derivatives,[16,,§,7].,Possibility,3,may,be,made,explicit,by,introducing,the,Lagrangian,density,(cid:96),which,is,a,functional,of,a,vector,ﬁeld,v,and,a,scalar,ﬁeld,s,,giving,an,action,(cid:90),L,=,(cid:96)(v,,s)dxdt,The,resulting,variational,principle,δL,=,0,is,δL,=,(cid:90),(cid:18),δ(cid:96),δv,·,δv,+,δ(cid:96),δs,(cid:19),δs,dxdt,=,0,6,(8),(9),(12),(13),(14),(15),This,must,hold,for,variations,η(r,,t),in,position,r(t),of,the,ﬂuid,elements,,which,are,arbitrary,except,that,they,must,vanish,at,the,endpoints,of,the,time,interval.,Suppose,the,perturbations,emerge,as,parameter,(cid:15),increases,from,zero,,as,∂r,∂(cid:15),∂r,∂t,=,η(r,,t,,(cid:15)),=,v(r,,t,,(cid:15)),(10),(11),Assuming,r(t,,(cid:15)),and,using,Equations(10),and,(11),,the,velocity,variations,are,found,to,be,con-,strained,by,and,correspondingly,scalar,variations,by,δv,=,∂η,∂t,+,Lv(η),,where,the,Lie,derivative,of,vector,w,is,δs,=,−Lη(s),Lv(w),=,−v,·,∇w,+,w,·,∇v,=,[v,,w],(Lie,bracket),and,the,Lie,scalar,derivative,is,Lη(s),=,η,·,∇s,A,point,to,note,is,that,should,variation,involve,changes,to,a,pseudsoscalar,such,as,magnetic,ﬁeld,B,or,a,pseudoscalar,such,as,density,ρ,,then,different,deﬁnitions,of,Lie,derivative,apply,,viz.,L2,η(B),=,+η∇,·,B,−,∇,×,(η,×,B),L3,η(ρ),=,∇,·,(ηρ),(16),(17),There,is,unfortunately,the,restriction,to,models,of,magnetic,ﬁeld,and,thermodynamic,evolution,which,are,non-dissipative.,Although,for,example,,Salmon,[17],points,out,that,dissipative,effects,can,also,be,treated,within,a,variational,framework,,albeit,at,the,expense,of,solving,for,additional,ﬁeld,variables,,it,is,unclear,how,this,might,be,used,here,,except,as,a,guide,as,to,the,effects,might,best,be,discretised.,However,,it,seems,the,above,variational,principle,provides,little,useful,guidance,as,to,how,the,non-,dissipative,effects,might,be,discretised.,The,natural,approach,to,the,discretisation,of,Equation,(9),would,be,to,assume,that,the,variations,are,speciﬁed,by,weighted,sums,of,elements,of,a,basis,function,space,say,φpqr,,so,that,Equation,(9),yields,a,set,of,equations,,one,for,each,member,of,the,function,space.,The,problem,arises,from,the,need,to,have,a,Lagrangian,density,(cid:96),=,1,2,ρv2,to,model,compressible,ﬂow.,Evidently,,the,variational,equation,contains,separate,terms,corresponding,to,variation,of,v,and,ρ,respectively,,namely,(cid:90),(cid:18),ρv,·,δv,+,δL,=,(cid:19),v2δρ,1,2,dxdt,=,0,(18),Substituting,for,v,using,Equation,(12),and,δρ,=,−L3,η(ρ),(Equation,(17)),it,is,evident,that,the,two,separate,terms,in,Equation,(18),have,to,partially,cancel,analytically,in,order,to,give,the,expected,ρv,·,∇v,term.,Thus,unless,care,is,taken,,the,natural,approach,to,discretisation,may,give,rise,to,7,additional,spurious,terms.,If,the,standard,analytic,manipulations,are,pursued,,the,result,is,the,usual,ideal,ﬂuid,evolution,equations,weighted,by,η,integrated,over,time,and,space.,It,is,interesting,that,the,implied,time,weight,function,has,to,vanishes,at,the,end-points,of,the,range,of,integration,,but,really,nothing,useful,for,discretisation,appears,to,arise,from,variational,approaches,,and,it,seems,preferable,to,continue,to,discretise,the,hydrodynamic,equations,directly.,4,Modelling,with,spectral,elements,This,section,contains,material,relevant,to,the,ongoing,re-shaping,of,Nektar++,into,a,FEM,li-,brary,designed,to,suit,NEPTUNE,requirements,,particularly,regarding,interfacing,with,UKAEA-,developed,particle,codes.,In,order,to,clarify,the,scope,of,future,work,,some,attention,is,given,to,accurate,description,of,what,is,currently,available,within,the,code.,4.1,Refactoring,and,extension,of,Nektar++,Efforts,to,prepare,Nektar++,for,the,Exascale,are,vital,for,NEPTUNE,purposes,and,currently,con-,stitute,an,effort,to,make,the,code,compatible,,in,terms,of,performance,and,scalability,,with,large,GPU-based,systems,(there,is,clearly,strong,developer,motivation,for,this,modiﬁcation,as,well,,given,that,the,full,DNS,CFD,simulation,of,a,realistic,system,of,industrial,interest,is,often,an,exas-,cale,problem).,Much,of,the,current,Nektar++,code,base,was,written,using,a,coarse-grained,MPI,parallelism,paradigm,without,support,for,vectorized,arithmetic.,GPU,support,is,presently,limited,to,speciﬁc,code,kernels,and,as,such,involves,signiﬁcant,data,transfer,workload,on,and,off,the,GPU,and,also,run-time,reconﬁguration,of,data,structures.,A,unifying,strategy,for,an,efﬁcient,GPU,implementation,involves,separation,of,the,data,storage,from,the,operators;,hence,,the,current,ExpList,class,is,intended,to,transition,to,a,pure,storage,object,,called,FieldStorage,(NB.,more,recently,renamed,to,just,Field),,which,will,be,device-aware,and,capable,of,supporting,a,range,of,data,layouts,(and,which,may,ultimately,use,a,SYCL,buffer).,In,addition,,core,operators,will,support,CPUs,(x86,and,ARM,,along,with,AVX2,/,AVX512,and,equivalents),and,GPUs.,There,are,many,other,details,in,the,ongoing,development,plan,eg.,ensuring,data,locality,in,discontinuous,Galerkin,implementations,,avoiding,retaining,in-memory,unnecessary,objects,such,as,pre-static-,condensation,version,of,system,matrices,,and,avoiding,the,use,of,shared,pointers,for,small,objects,(which,wastes,execution,time).,These,refactoring,plans,are,in,tandem,with,a,migration,to,C++17.,The,Nektar++,Python,interface,is,also,being,upgraded,in,scope,from,a,workﬂow,automation,utility,to,a,more,general,interface,,leveraging,the,suitability,of,Python,as,a,‘glue’,language.,Functionality,will,include,the,programmatic,speciﬁcation,of,simulations,(bypassing,the,current,requirement,for,an,XML,session,ﬁle),and,wrapping,sufﬁcient,API,functionality,to,facilitate,solver,integration,with,other,software,(note,the,ubiquity,of,Python,in,user,interfaces),and,also,enable,the,construction,of,new,solvers,within,Python,in,a,DSL-like,manner,that,avoids,the,need,to,recompile,the,library,to,generate,a,solver,(ie.,the,‘outer,loop’,can,be,Python),or,implement,a,novel,operator.,The,use,of,implicit,time-stepping,techniques,has,been,noted,as,an,area,of,great,relevance,for,NEPTUNE.,It,has,been,noted,that,some,Nektar++,incompressible,Navier-Stokes,simulations,are,constrained,by,stability,to,use,a,time,step,that,may,be,up,to,two,orders,of,magnitude,below,that,8,needed,to,resolve,the,physics,of,interest,-,this,is,a,problem,that,becomes,worse,the,smaller,the,element,size,in,the,mesh.,Recent,developments,in,implementing,a,fully-implicit,scheme,(based,on,[18]),were,presented,at,the,recent,Nektar++,Workshop,2022,and,showed,promising,initial,results,with,an,order,of,magnitude,increase,in,stable,time,step,size.,Note,that,the,current,scheme,is,mixed,with,the,diffusion,treated,implicitly,and,the,advection,explicitly,in,an,IMEX,time-advancing,scheme;,the,explicit,advection,means,that,the,code,is,subject,to,a,maximum,stable,time,step,size,from,the,CFL,criterion.,This,work,has,the,potential,to,speed,up,the,simulations,of,convective,heat,transfer,reported,in,[19],and,may,apply,more,generally,to,future,NEPTUNE,simulations.,4.2,Meshing,A,number,of,additions,have,been,made,to,the,2-D,meshing,capabilities,of,NekMesh,under,the,programme,of,work,in,grant,T/NA078/20.,These,include,the,ability,to,perform,r-,and,h-reﬁnement,,or,the,generation,of,a,quadrilateral-meshed,‘boundary,layer’,region,,local,to,a,particular,CAD-exact,curve,bounding,the,mesh,in,order,to,achieve,a,higher,numerical,resolution,in,the,neighbourhood,of,a,particular,magnetic,ﬁeld,surface.,The,magnetic,ﬁeld,surfaces,in,a,tokamak,are,three-dimensional,so,it,will,be,necessary,to,provide,similar,capabilities,in,three-dimensional,meshes,,initially,in,the,case,of,axisymmetric,geometries.,Another,meshing,issue,,which,arises,from,the,need,to,integrate,Nektar++,with,a,particle-in-cell,code,,is,the,need,to,generate,non-physical,‘halo’,(aka,‘guard’),cells,surrounding,the,sub-meshes,that,are,generated,when,a,large,computational,mesh,is,distributed,over,potentially,a,large,number,of,MPI,ranks.,These,‘halo’,cells,are,necessary,for,keeping,track,of,particles,transitioning,between,MPI,ranks.,It,is,envisaged,that,this,functionality,will,be,added,to,NekMesh.,A,ﬁnal,note,concerns,the,higher-dimensional,representation,of,non-equilibrium,continuum,ﬁelds,,which,is,an,alternate,choice,to,a,particle-based,scheme.,This,involves,meshes,in,up,to,three,additional,dimensions,(velocity-space).,It,has,become,clear,that,the,meshing,of,velocity,space,is,non-trivial,because,of,the,existence,of,eg.,outﬂow,boundary,conditions,in,the,Parra,et,al,formula-,tion,[1],and,so,it,is,not,possible,in,all,cases,to,mesh,velocity,space,with,a,uniform,grid.,4.3,Incorporation,of,ﬁnite,element,exterior,calculus,(FEEC),in,Nektar++,The,use,of,techniques,from,ﬁnite,element,exterior,calculus,(for,which,see,[20]),is,seen,as,one,of,the,main,pathways,to,an,effective,code,that,couples,particles,to,an,electromagnetic,ﬁeld.,The,necessary,element,types,(here,called,‘conforming’),and,mathematical,framework,(for,example,,the,weak,mixed,formulation,of,a,partial,differential,equation),are,not,yet,implemented,within,Nek-,tar++.,(Note,,however,,that,the,developers,have,considered,the,appropriateness,of,the,numerical,schemes,where,necessary,eg.,in,the,incompressible,Navier-Stokes,solver,it,is,possible,to,specify,a,Taylor-Hood,scheme,known,to,give,a,stable,discretization,,in,which,the,velocity,components,are,discretized,with,a,polynomial,space,one,order,higher,than,that,used,for,the,pressure.),The,classes,of,ﬁnite,element,currently,implemented,in,Nektar++,are,listed,below,(and,see,p.50,onward,of,[7]),,in,which,the,nomenclature,is,as,in,a,Nektar++,session,ﬁle.,The,bases,divide,into,two,types:,modal,,and,nodal.,The,elements,are,generically,implemented,by,mapping,the,physical,9,element,to,a,reference,element,,which,is,typically,deﬁned,on,the,coordinate,interval,[−1,,1],per,dimension.,All,elements,store,scalar,degrees,of,freedom,and,there,is,currently,no,model,for,vector,or,tensor,quantities,within,the,core,Nektar++,libraries.,•,MODIFIED:,a,modiﬁed,orthogonal,polynomial,basis,,with,only,two,modes,per,dimension,being,non-zero,on,the,element,boundaries.,Aside,from,this,criterion,,which,reduces,the,density,of,the,coupling,between,ﬁnite,elements,,the,basis,functions,are,not,localized,to,particular,nodes.,The,basis,is,hierarchical,,meaning,that,increasing,order,p,−,1,to,p,requires,only,the,addition,of,order-p,modes,rather,than,the,reassignment,of,the,entire,basis,and,that,lower-,order,computations,are,a,subset,of,a,given,order.,The,basis,functions,in,one,dimension,are,,for,order,p,and,ξ,∈,[−1,,1],,ψn(ξ),=,,,,1−ξ,2,(cid:16),1−ξ,2,1+ξ,2,(n,=,0),,(cid:17),(cid:17),(cid:16),1−ξ,2,(n,=,p).,P,1,1,n−1(ξ),,n,=,1,,.,.,.,,,p,−,1,(19),The,P,α,β,m,(ξ),denotes,the,order-m,Jacobi,polynomial,(the,case,α,=,β,=,0,corresponds,to,the,familiar,Legendre,polynomials,and,the,α,=,β,=,1,case,used,here,to,their,derivatives).,This,choice,maintains,a,high,degree,of,orthogonality,,for,example,the,elemental,mass,matrix,has,a,mostly,tri-diagonal,structure,and,the,Laplacian,matrix,has,,except,for,the,boundary,modes,,a,diagonal,structure.,•,GLL,LAGRANGE:,an,interpolatory,and,therefore,nodal,basis,in,which,a,Gauss-Lobatto-Legendre,(GLL),quadrature,rule,with,p+2,points,is,used,,giving,an,exact,integration,of,the,mass,matrix,,which,is,full.,•,GLL,LAGRANGE,SEM:,an,interpolatory,and,therefore,nodal,basis,in,which,a,reduced,GLL,quadra-,ture,rule,with,p,+,1,points,is,used;,the,quadrature,points,are,collocated,with,the,Lagrange,nodes.,Without,lumping,,this,results,in,a,diagonal,mass,matrix,with,entries,that,are,the,quadrature,weights.,This,is,the,‘classical’,spectral,element,method.,Both,GLL,schemes,are,nodal,bases,in,which,the,Lagrange,polynomials,are,deﬁned,with,points,at,the,zeros,of,the,Gauss-Lobatto,polynomials.,The,only,difference,lies,in,the,quadrature,rule:,a,GLL,quadrature,with,p,+,1,points,is,exact,only,up,to,and,including,polynomials,of,order,2p,−,1,whereas,the,mass,matrix,,being,bilinear,in,the,basis,functions,,contains,integrals,of,polynomials,of,order,2p.,Section,2.3,of,ref,[7],indicates,that,the,error,in,the,inexact,diagonal,lumped,mass,matrix,is,of,the,same,order,as,the,approximation,error,of,the,expansion.,The,above,elements,are,implemented,in,continuous,Galerkin,on,2-D,quadrilaterals,and,3-D,hex-,ahedra,and,also,,with,some,exceptions,,simplices,(2-D,triangles,,3-D,tetrahedra);,discontinuous,Galerkin,is,available,with,the,MODIFIED,basis,for,all,shape,types,but,only,quads,/,hexes,for,the,nodal,bases,[21].,In,addition,,it,is,possible,to,specify,different,polynomial,orders,in,different,dimen-,sions,for,tensor,product,elements,as,well,as,different,point,sets,for,the,locations,of,pointwise,ﬁeld,values.,Some,of,the,important,element,types,used,in,FEEC,store,vectorial,degrees,of,freedom,of,various,types,(eg.,the,H(div)-conforming,elements,have,degrees,of,freedom,corresponding,to,the,vector,10,ﬁeld,component,normal,to,the,element,boundaries,,and,the,H(curl)-conforming,elements,store,components,tangent,to,element,edges).,Nektar++,currently,represents,vectors,on,a,scalar-per-,Cartesian,component,basis,and,so,it,seems,that,the,vectorial,element,classes,may,bring,additional,beneﬁt,in,terms,of,the,amount,of,data,stored,(viz.,store,the,component,along,an,edge,,rather,than,three,independent,Cartesian,components).,It,is,not,known,to,the,author,whether,eg.,the,reduced,quadrature,technique,of,classical,spectral,elements,can,be,applied,to,the,ﬁnite,element,types,used,in,FEEC.,4.3.1,Transformation,from,reference,element,One,particular,issue,in,implementing,conforming,discretizations,(eg.,the,Raviart-Thomas,elements,[22],,which,discretize,H(div)),in,Nektar++,lies,with,the,strategy,of,mapping,quantities,from,a,ref-,erence,element,onto,the,general,ﬁnite,element,in,the,mesh.,This,technique,enables,the,precom-,putation,of,quadratures,in,the,reference,space,(ie.,once),in,order,to,construct,element,matrices,which,are,then,mapped,to,each,of,the,‘physical’,ﬁnite,elements.,A,lucid,presentation,is,found,in,[23],in,which,is,discussed,assembly,of,H(div),and,H(curl),ﬁnite,elements,,in,the,context,of,the,FEniCS,project,[24].,For,scalar,quantities,the,transformation,law,applicable,to,the,evaluation,of,an,integral,over,a,ﬁnite,element,is,trivial:,F,scalar(Φ),=,Φ,◦,F,−1.,(20),the,content,of,which,may,be,illustrated,as,follows.,The,mapping,F,of,coordinates,,for,a,simplex,in,two,dimensions,,where,the,target,triangle,has,vertex,coordinates,(xi,,yi),for,i,=,1,,2,,3,,in,the,afﬁne,case,is,given,by,x,=,r(x2,−,x1),+,s(x3,−,x1),,y,=,r(y2,−,y1),+,s(y3,−,y1),(21),(22),Thus,another,way,of,regarding,Equation,(20),is,that,on,the,left,,is,the,scalar,function,Φ,as,a,function,of,‘real’,space,coordinates,(x,,y),and,on,the,right,as,a,function,of,local,coordinates,(r,,s),,so,that,F,scalar(Φ),=,Φ(x(r,,s),,y(r,,s)).,The,Jacobian,matrix,of,F,is,simply,J,=,(cid:18)x2,−,x1,x3,−,x1,y3,−,y1,y2,−,y1,(cid:19),.,(23),It,is,apparent,that,the,above,transformation,law,does,not,in,general,preserve,the,continuity,of,nor-,mal,or,tangential,traces,that,would,be,necessary,for,a,continuous,mapping,of,vector,divergences,or,vector,curls,,otherwise,respectively,,H(div,,Ω0),→,H(div,,Ω),or,H(curl,,Ω0),→,H(curl,,Ω),(where,Ω0,denotes,the,reference,element,and,Ω,the,physical,element).,The,normal,trace,continuity,is,preserved,by,the,contravariant,Piola,mapping,F,div(Φ),=,1,det(J),JΦ,◦,F,−1,,11,(24),while,the,tangent,trace,continuity,is,preserved,by,the,covariant,Piola,mapping,F,curl(Φ),=,J,−T,Φ,◦,F,−1.,(25),Note,that,in,two,dimensions,,the,contravariant,and,covariant,Piola,transforms,are,equivalent,in,the,case,of,a,conformal,,orientation-preserving,map,,which,is,easily,shown,using,the,Cauchy-Riemann,equations,to,equate,components,in,the,following:,1,det(J),J,=,1,det(J),(cid:32),∂u,∂x,∂v,∂x,(cid:33),∂u,∂y,∂v,∂y,and,J,−T,=,1,det(J),(cid:32),∂v,∂y,−,∂v,(cid:33),.,∂x,∂u,∂x,−,∂u,∂y,(26),(27),It,would,be,necessary,to,write,implementations,of,the,transformations,(24),,25,in,order,to,provide,Nektar++,implementations,of,the,relevant,elements.,An,initial,version,could,support,a,restricted,range,of,operators,,avoiding,some,of,the,complexity,of,a,ﬂexible,form,language,implementation,as,in,[23].,Note,that,the,transformations,to,reference,element,are,more,involved,for,the,non-straight-sided,el-,ements,that,are,likely,to,occur,when,using,meshes,that,conform,to,the,ﬁrst,wall,or,to,magnetic,ﬂux,surfaces,,though,the,results,that,the,Piola,transformations,preserve,the,stated,trace,continuities,still,hold,for,smooth,nondegenerate,mappings,where,the,Jacobian,matrix,is,invertible.,4.3.2,Sum-factorization,The,sum-factorization,or,tensor,product,technique,was,re-discovered,by,Orszag,[25],and,is,con-,sidered,to,be,the,key,to,the,efﬁciency,of,spectral,methods.,It,relies,on,the,expansions,having,a,tensor,product,decomposition.,A,description,can,be,found,on,[7,,p.179],,in,which,the,central,formula,is,Uij,=,p,(cid:88),p,(cid:88),r=1,s=1,frshirhjs,≡,p,(cid:88),r=1,hir,(cid:32),p,(cid:88),s=1,(cid:33),frshjs,,,(28),the,key,observation,being,that,the,bracket,on,the,in,the,right-hand-side,expression,is,independent,of,i,and,so,the,bracket,is,computed,once,for,each,choice,of,r,,j,and,then,re.used,for,each,i.,This,reduces,the,O(p4),complexity,of,the,double,sum,and,the,evaluations,over,i,,j,to,two,O(p3),summations.,The,double,sum,in,(28),is,appropriate,to,2-D,reductions,but,recursive,application,of,the,same,principle,leads,to,further,efﬁciencies,in,higher,dimensions;,generally,it,is,possible,to,reduce,the,algorithmic,complexity,of,operations,such,as,the,above,from,p2D,to,pD+1,where,D,is,the,number,of,space,dimensions.,Note,there,must,also,be,a,tensor,product,structure,in,the,12,computed,quantity,ie.,it,is,labelled,by,i,and,j,and,there,is,no,gain,,for,example,,for,the,repeated,evaluation,of,a,tensor-product,ﬁeld,at,a,set,of,O(p),arbitrary,points,(in,this,case,there,is,a,δij,on,both,sides,of,the,equation,and,the,sums,are,already,O(p3)).,There,are,a,number,of,deployments,of,this,tactic,within,Nektar++,,for,example,the,backward,trans-,formation,which,corresponds,to,evaluation,of,the,ﬁeld,at,quadrature,points,starting,from,the,coef-,ﬁcients,of,the,expansion,basis,(ie.,a,change,of,basis),and,is,called,as,BwdTrans,in,the,API,-,see,implementations,for,various,element,types,eg.,Nektar::Collections::BwdTrans,SumFac,Tri,,Nektar::Collections::BwdTrans,SumFac,Hex,(see,[26]).,The,inner,product,is,another,example,(see,eg.,Nektar::Collections::IProductWRTBase,SumFac,Quad).,It,is,also,obvious,that,the,technique,applies,to,the,interpolation,of,ﬁeld,data,onto,a,rectilinear,grid,,perhaps,for,producing,a,visual,display,of,the,spatial,ﬁeld,or,a,representation,suitable,for,data,processing,eg.,spatial,Fourier,transform.,The,example,in,Appendix,B,shows,a,similar,sort,of,efﬁciency,gain.,It,will,be,essential,to,leverage,this,technique,,wherever,applicable,,in,any,new,code.,The,paper,[27],contains,description,of,sum,factorization,on,cuboid,cells,for,H(div),and,H(curl),conforming,elements,within,the,Firedrake,framework,[28],.,The,potential,applicability,of,sum,factorization,for,conforming,element,types,on,simplices,is,currently,less,clear;,an,initial,implementation,could,be,restricted,to,regular,meshes,and,slab,geometries,in,NEPTUNE.,13,5,Summary,Spectral,element,models,implemented,in,the,Nektar++,framework,continue,to,be,central,to,future,plans,for,NEPTUNE.,There,is,clear,evidence,that,the,Nektar++,developers,have,designs,to,en-,sure,that,their,code,can,continue,to,prosper,in,a,GPU-dominated,exascale,era,and,also,to,improve,the,user,experience,regarding,adding,new,solvers,and,interfacing,with,other,codes,by,means,of,the,Python,interface.,Material,in,this,report,has,been,written,with,the,aim,of,indicating,potential,issues,in,implementing,ﬁnite,element,exterior,calculus,techniques,in,Nektar++,with,reference,to,publications,by,the,developers,of,FEniCS,/,Firedrake.,The,extent,to,which,these,methods,syn-,chronize,with,the,existing,efﬁciency,optimizations,of,Nektar++,(chieﬂy,,sum,factorization),remains,to,be,fully,explored.,As,far,as,the,overall,project,NEPTUNE,is,concerned,,resolving,the,numerical,difﬁculties,encoun-,tered,in,the,work,of,Grant,T/NA085/20,ought,to,be,the,priority,objective.,There,is,a,lack,of,detail,at,present,to,how,their,root,cause,might,be,identiﬁed,and,the,difﬁculties,overcome,,but,assuming,the,root,cause,to,be,the,discontinuous,nature,of,the,distribution,function,at,the,effective,wall,,then,there,are,range,of,subgridding,techniques,,most,notably,artiﬁcial,viscosities,such,as,SVV,,‘Spec-,tral,Vanishing,Viscosity’,(Tadmor,,as,referenced,in,the,textbook,by,Karniadakis,and,Sherwin,[7]),to,mollify,the,Gibbs’,phenomenon.,However,,other,causes,may,necessitate,the,use,of,ideas,from,moving,boundary,numerics,,mappings,,and/or,immersing,boundaries.,Acknowledgement,The,support,of,the,UK,Meteorological,Ofﬁce,and,Strategic,Priorities,Fund,is,acknowledged.,References,[1],J.,T.,Parker,and,W.,Arter.,Technical,report,on,Physics,model,Technical,Report,CD/EXCALIBUR-FMS/0058-M2.8.2,,UKAEA,,//github.com/ExCALIBUR-NEPTUNE/Documents/blob/main/reports/ukaea_reports/,CD-EXCALIBUR-FMS0058-M2.8.2.pdf.,2021.,selection.,https:,[2],J.W.,Eastwood,and,W.,Arter.,Interpretation,of,disruptions,in,tokamak,simulations.,Physical,Review,Letters,,57:2528–2531,,1986.,http://dx.doi.org/10.1103/PhysRevLett.57.2528.,[3],J.W.,Eastwood,and,W.,Arter.,Spurious,behaviour,of,numerically,computed,ﬂuid,ﬂow.,I.M.A.,Journal,of,Numerical,Analysis,,7:205–222,,1987.,[4],D.J.,Acheson.,Local,analysis,of,thermal,and,magnetic,instabilities,in,a,rapidly,rotating,ﬂuid.,Geophysical,&,Astrophysical,Fluid,Dynamics,,27(1-2):123–136,,1983.,[5],W.O.,Criminale,,T.L.,Jackson,,and,R.D.,Joslin.,Theory,and,Computation,in,Hydrodynamic,Stability.,Cambridge,University,Press,,2003.,14,[6],J.P.,Boyd.,Chebyshev,and,Fourier,Spectral,Methods.,Dover,Publications,,Mineola,,NY,,2001.,http://www-personal.umich.edu/,jpboyd/BOOK,Spectral2000.html.,[7],G.,Karniadakis,and,S.,Sherwin.,Spectral/hp,element,methods,for,computational,ﬂuid,dy-,namics,2nd,Ed.,Oxford,University,Press,,2005.,https://doi.org/10.1093/acprof:oso/,9780198528692.001.0001.,[8],Linda,E,Reichl.,A,Modern,Course,in,Statistical,Physics,4th,Ed.,John,Wiley,&,Sons,,2016.,[9],P.,Helander,and,D.J.,Sigmar.,Collisional,transport,in,magnetized,plasmas.,Cambridge,Uni-,versity,Press,,2005.,Equations,[10],W.,Arter.,Report,github.com/ExCALIBUR-NEPTUNE/Documents/blob/main/reports/ukaea_reports/,CD-EXCALIBUR-FMS0021-1.01-M1.2.1.pdf.,for,EXCALIBUR/NEPTUNE,Proxyapps.,CD/EXCALIBUR-FMS/0021-1.20-M1.2.1,,UKAEA,,2021.,Technical,https://,[11],W.,Arter.,Numerical,simulation,of,magnetic,fusion,plasmas.,Reports,on,Progress,in,Physics,,58:1–59,,1995.,http://dx.doi.org/10.1088/0034-4885/58/1/001.,[12],Code,generation,QPROG,style,,example,of,Braginskii,plasma,transport,coefﬁcients.,https:,//github.com/wayne-arter/smardda-misc,,2017.,Accessed:,December,2020.,[13],J.,Larsson.,A,practical,form,of,Lagrange–Hamilton,theory,for,ideal,ﬂuids,and,plasmas.,Journal,of,Plasma,Physics,,69(3):211–252,,2003.,[14],D.,Lovelock,and,H.,Rund.,Tensors,,Differential,Forms,and,Variational,Principles.,Dover,,New,York,,1988.,[15],G.,Webb.,Magnetohydrodynamics,and,Fluid,Dynamics:,Action,Principles,and,Conservation,Laws.,Springer,,2018.,[16],D.D.,Holm,,J.E.,Marsden,,and,T.S.,Ratiu.,The,Euler,Poincare,Equations,and,Semidirect,Products,with,Applications,to,Continuum,Theories.,Advances,in,Mathematics,,137:1–81,,1998.,[17],R.,Salmon.,Hamiltonian,ﬂuid,mechanics.,Annual,Review,of,Fluid,Mechanics,,20:225–256,,1988.,[18],Dong,S.,and,Shen,J.,An,unconditionally,stable,rotational,velocity-correction,scheme,for,in-,compressible,ﬂows.,Journal,of,Computational,Physics,229,(2010),,7013-7029,,2010.,[19],E.,Threlfall,and,W.,Arter.,Finite,Element,Models:,I.,Technical,Report,CD/EXCALIBUR-FMS/0051-M6.1,,UKAEA,,2021.,//github.com/ExCALIBUR-NEPTUNE/Documents/blob/main/reports/ukaea_reports/,CD-EXCALIBUR-FMS0051-M6.1.pdf.,Complementary,Activities,https:,[20],D.N.,Arnold.,Finite,Element,Exterior,Calculus.,CBMS-NSF,regional,conference,series,in,applied,mathematics,93,,SIAM,,2018.,[21],Moxey,D.,Private,communication,,2022.,15,[22],Raviart-Thomas,ﬁnite,elements.,https://en.wikipedia.org/wiki/Raviart-Thomas_,basis_functions.,Accessed:,September,2022.,[23],Rognes,M.E.,,Kirby,R.C.,,and,Logg,A.,Efﬁcient,assembly,of,H(div),and,H(curl),conforming,ﬁnite,elements.,SIAM,Journal,on,Scientiﬁc,Computing,31,,no.,6,(2010):,4130-4151,,2010.,[24],FEniCS,Project.,https://fenicsproject.org/.,Accessed:,September,2022.,[25],Orszag,S.A.,Spectral,methods,for,problems,in,complex,geometries.,Journal,of,Computational,Physics,37,,70,,1980.,[26],Nektar++,documentation:,BdwTrans.cpp,ﬁle,reference.,https://doc.nektar.info/,doxygen/4.4.1/_bwd_trans_8cpp.html.,Accessed:,October,2022.,[27],Homolya,M.,,Kirby,R.C.,,and,Ham,D.A.,Exposing,and,exploiting,structure:,optimal,code,generation,for,high-order,ﬁnite,element,methods.,submitted,to,ACM,Transactions,on,Mathe-,matical,Software,,2017.,[28],Florian,Rathgeber,,David,A.,Ham,,Lawrence,Mitchell,,Michael,Lange,,Fabio,Luporini,,An-,drew,T.,T.,Mcrae,,Gheorghe-Teodor,Bercea,,Graham,R.,Markall,,and,Paul,H.,J.,Kelly.,Fire-,drake:,automating,the,ﬁnite,element,method,by,composing,abstractions.,ACM,Trans.,Math.,Softw.,,43(3):24:124:27,,2016.,URL:,http://arxiv.org/abs/1501.01809,,arXiv:1501.01809,,doi:10.1145/2998441.,,2015.,[29],Busch,K.,,K,¨onig,M.,,and,Niegemann,J.,Discontinuous,Galerkin,methods,in,nanophotonics.,Laser,Photonics,Rev.,5,,No.,6,,773809,(2011),/,DOI,10.1002/lpor.201000045,,2011.,[30],Niegemann,J.,,Diehl,R.,,and,Busch,K.,Efﬁcient,low-storage,Runge-Kutta,schemes,with,op-,timized,stability,regions.,Journal,of,Computational,Physics,231,Issue,2,(2012),,364-372,,2012.,A,Extracts,from,External,Grants,for,NEPTUNE,Y4-6,A.1,Spectral,element,procurement,Spectral,elements,will,form,the,core,of,the,NEPTUNE,software,,and,thus,any,usage,thereof,must,perform,well,at,the,Exascale.,Moreover,,the,elements,need,to,have,non-planar,surfaces,in,order,to,represent,more,efﬁciently,and,accurately,the,ﬁrst,wall,of,the,tokamak.,The,requirement,for,close,coupling,with,a,high-dimensional,representation,of,the,plasma,,such,as,either,eg.,by,particles,or,by,gyro-averaged,kinetic,equations,,represents,a,further,challenge.,A,speciﬁc,NEPTUNE,objective,which,must,be,supported,as,a,priority,is,the,production,of,a,2d3v,proxyapp.,to,be,helped,by,completing,Proxyapp,2-6,(described,under,number,6,in,FM-WP2),following,the,plan,set,out,in,a,report,to,be,produced,under,contract,T/NA083/20,or,otherwise.,There,will,therefore,be,an,ongoing,need,throughout,the,remainder,of,the,project,,to,support,usage,and,development,of,a,library,of,spectral,elements,,such,as,Nektar++.,Software,development,will,be,necessary,in,order,to,accommodate,efﬁcient,interaction,with,the,high-dimensional,representations,,16,and,likely,be,required,in,order,to,ensure,efﬁcient,operation,on,new,architectures,,indeed,for,long,term,code,stability,,it,would,be,desirable,to,refactor,the,library,to,minimise,dependencies,on,other,libraries.,There,will,be,a,similar,ongoing,need,to,support,usage,and,development,of,a,mesher,geared,to,the,production,of,non-planar,high,order,ﬁnite,elements,,such,as,Nekmesh.,The,outputs,will,therefore,be,the,core,of,an,efﬁcient,spectral,element,mesher,and,a,library,or,package,of,libraries,of,spectral,elements,tuned,for,use,at,Exascale,and,the,special,needs,of,project,NEPTUNE,,as,represented,by,higher,layer,components,corresponding,to,respectively,a,developer,DSL,(in,SYCL/C++),and,an,end-user,DSL,at,the,level,of,Python,or,Julia.,The,supporting,Nektar++,library/ies,must,be,demonstrated,in,particular,to,enable,efﬁcient,solution,for,Proxyapp,2-6,,for,the,2d3v,proxyapp,,and,for,3-D,elliptic,problems,at,Exascale.,Proxyapps,,to,include,Proxyapp,3-2,,should,conform,to,emerging,high,standards,for,the,production,of,NEPTUNE,software.,A.2,Advanced,Referent,Model,Procurement,The,objective,is,to,produce,a,gyro-averaged,theoretical,time,dependent,model,of,plasma,suitable,for,project,NEPTUNE,,meaning,that,it,is,of,actionable,accuracy,in,the,outer,pedestal,,separatrix,region,and,scrape-off,layers,of,a,tokamak,plasma.,Building,on,the,work,of,Grant,T/NA085/20,or,otherwise,,such,a,generally,5-D,model,(3-D,in,space,,2-D,in,velocity,space),is,likely,to,have,the,property,of,reducing,to,a,ﬂuid-like,(5-moment,,only,spatial,dependence),model,in,regions,of,higher,collisionality.,Such,a,model,is,expected,to,require,special,treatments,for,the,separatrix,and,for,the,wall,,in,order,to,account,for,realistic,magnetic,equilibria,and,ensure,accurate,modelling,of,the,geometry.,Non-axisymmetric,electromagnetic,ﬁeld,effects,,and,coupling,between,different,plasma,species,in,addition,to,neutral,components,should,also,be,considered.,The,output,will,be,theoretical,models,that,include,in,the,aforementioned,effects,,wherein,the,deriva-,tion,of,suitable,collision,operator,will,be,of,especial,importance.,The,practicality,of,these,models,should,be,demonstrated,by,suitable,proxyapps,,conforming,to,the,emerging,high,NEPTUNE,stan-,dard,where,appropriate.,B,Note,on,a,computationally-efﬁcient,discontinuous,Galerkin,method,In,this,section,a,simple,example,of,a,modal,time-evolution,code,for,Maxwell’s,equations,in,two,dimensions,is,exhibited,as,an,example,showing,the,types,of,structure,that,can,be,exploited,for,numerical,optimizations.,The,implementation,is,a,time-explicit,upwinded,discontinuous,Galerkin,scheme;,for,more,details,,albeit,with,a,focus,on,nodal,methods,,see,[29].,The,method,shows,re-,duction,in,algorithmic,complexity,(over,a,naive,implementation),equivalent,to,that,in,the,description,of,sum-factorization,in,the,main,text.,For,simplicity,,assume,a,uniform,grid,of,square,elements.,The,mapping,between,the,reference,elements,and,the,physical,element,is,a,trivial,rescaling.,The,hierarchical,basis,functions,per,dimension,are,the,Legendre,polynomials,ψn(ξ),=,P,0,0,n,(ξ),on,the,interval,ξ,∈,[−1,,1],and,the,normalization,of,the,two-dimensional,tensor,product,modes,is,such,that,17,(cid:90),1,(cid:90),1,−1,−1,ψm(x,,y)ψn(x,,y)d2x,=,δmn.,This,means,that,the,element,mass,matrix,is,explicitly,a,multiple,of,the,identity,matrix.,Other,matrices,required,for,the,DG,scheme,are,the,element,stiffness,matrices,(Sx)mn,=,(Sy)mn,=,(cid:90),1,(cid:90),1,−1,(cid:90),1,−1,(cid:90),1,−1,−1,ψm(x,,y),ψm(x,,y),∂,∂x,∂,∂y,ψn(x,,y)d2x,,ψn(x,,y)d2x,,and,the,element,face,mass,matrices,are,,for,example,(Fy+)mn,=,(cid:90),1,(cid:90),1,−1,−1,ψm(x,,y)ψn(x,,y)δ(y,−,1)d2x.,(29),(30),(31),(32),These,need,to,be,evaluated,on,the,appropriate,edge,(ie.,subspace,of,codimension,one),ie.,at,a,ﬁxed,value,of,one,of,the,coordinates,as,indicated,by,the,delta,function.,B.1,One-dimensional,implementation,Maxwell’s,equations,in,vacuo,are,,in,natural,units,,˙E,=,∇,×,H,,˙H,=,−∇,×,E.,Specializing,to,one,dimension,(x),,these,become,˙Ey,=,−∂xHz,,˙Hz,=,−∂xEy.,(33),(34),(35),(36),The,computational,mesh,is,the,interval,[0,,1],divided,into,N,equal,elements,with,a,periodic,condi-,tion,at,the,interval,ends.,The,basis,is,Legendre,function,per,element,and,so,the,two,lowest,modes,are,,on,element,labelled,by,n,where,n,=,0,,1,,...,,N,−,1,,ψn,0,(x),=,ψn,1,(x),=,√,√,N,(cid:32),12N,3,x,−,(cid:33),.,n,+,1,2,N,18,(37),(38),These,are,normalized,such,that,(cid:82),local,(and,global),mass,matrix.,n,ψn,a,(x)ψn,b,(x)dx,=,δab,where,(cid:82),n,≡,(cid:82),n+1,N,n,N,ie.,to,give,an,identity,The,element,stiffness,matrix,has,one,non-zero,component,(Sx)01,≡,(cid:90),n,ψn,0,(x),∂,∂x,ψn,1,(x)dx,=,√,12N.,The,edge,mass,matrices,are,computed,as,(integration,over,point,is,point,evaluation),(Fx−),=,ψn,(Fx+),=,ψn,a,(x)ψn,a,(x)ψn,b,(x)|x=,n,,,b,(x)|x=,n+1,N,N,.,Hence,and,(Fx−)00,=,N,,√,(Fx−)01,=,−,(Fx−)10,=,−,(Fx−)11,=,3N.,√,3N,,3N,,(Fx+)00,=,N,,√,(Fx+)01,=,(Fx+)10,=,(Fx+)11,=,3N.,√,3N,,3N,,The,initial,condition,is,a,cosine,standing,wave,Ey,=,0,,Hz,=,cos,2πx.,(39),(40),(41),(42),(43),(44),(45),(46),(47),(48),(49),(50),(51),(52),(53),The,required,initial,data,are,overlap,integrals,between,this,initial,condition,and,the,individual,modes.,The,upwind,numerical,ﬂuxes,to,be,applied,to,˙Ey,and,˙Hz,respectively,are,19,(∆E,−,n(n,·,∆E),+,n,×,∆H,2,(∆H,−,n(n,·,∆H),−,n,×,∆E,2,,,.,The,left-hand-side,ﬂuxes,evaluate,to,and,the,right-hand-side,ﬂuxes,∆Ey,+,∆Hz,2,∆Hz,+,∆Ey,2,,,,,∆Ey,−,∆Hz,2,∆Hz,−,∆Ey,2,,,.,(54),(55),(56),(57),(58),(59),The,ﬁeld,differences,eg.,∆Ey,are,always,‘outside,minus,inside’,with,respect,to,the,element,in,question.,If,the,system,is,zeroth-order,,the,situation,is,very,simple.,There,is,no,stiffness,matrix,and,the,semi-discrete,equations,take,the,form,˙En,=,˙Hn,=,N,2,N,2,(En+1,+,En−1,−,2En,+,Hn−1,−,Hn+1),,,(60),(Hn+1,+,Hn−1,−,2Hn,+,En−1,−,En+1),.,where,N,appears,where,it,is,more,usual,to,see,1/h.,The,ﬁrst,3,terms,may,be,recognised,as,the,simplest,ﬁnite,difference,approximation,to,a,diffusion,term,h∂2/∂x2.,This,is,an,example,of,a,ﬁnite-,volume,method,(which,here,corresponds,to,zero-order,DG;,note,that,higher-order,extensions,of,ﬁnite-volume,methods,,eg.,the,Godunov,scheme,,do,exist).,The,1-D,code,can,be,run,with,a,choice,of,time,stepper,(a,14-stage,5th,LSRK,scheme,noted,for,its,good,stability,properties,is,used,[30]).,Outputs,from,the,code,can,be,seen,in,Fig.1.,Note,that,it,is,expected,that,an,upwind,DG,code,is,numerically,dissipative,,hence,the,loss,of,amplitude,seen,in,the,ﬁgure,(the,dissipation,diminishes,spectrally,as,the,order,is,increased).,B.2,Two-dimensional,implementation,Computation,of,the,relevant,matrices,makes,clear,the,efﬁciency,improvements,of,this,scheme,over,eg.,a,nodal,scheme,in,which,all,the,matrices,are,full.,Note,that,basis,in,2-D,falls,into,the,20,Figure,1:,Output,of,the,magnetic,ﬁeld,from,the,one-dimensional,DG,code,after,one,optical,cycle,,after,which,the,analytic,solution,has,returned,to,the,cosine,initial,condition.,This,is,seen,to,be,approximately,true,for,the,numerical,solutions,,however,,the,40-element,zero-order,code,output,(blue,curve),has,decayed,in,amplitude,due,to,the,dissipative,nature,of,the,upwinded,scheme.,The,ten-element,ﬁrst-order,code,(orange,curve),produces,a,much,better,answer.,Note,the,evident,ﬁeld,discontinuities,at,element,boundaries,in,both,examples.,21,familiar,triangular,binomial,pattern,(with,the,number,of,modes,needed,for,order-p,being,the,pth,triangular,number);,tiers,in,this,triangle,contain,modes,of,equivalent,aggregate,polynomial,order.,This,is,exhibited,in,Table,1,which,shows,the,ten,modes,needed,for,p,=,3,in,2-D;,all,are,deﬁned,on,(x,,y),∈,[−1,,1]2.,The,equivalent,basis,in,3-D,has,a,‘trinomial’,structure,(and,the,total,number,of,modes,is,a,tetrahedal,number).,mode,number,i,0,1,2,3,4,5,6,7,8,9,tier,mode,num,x,mode,num,y,0,1,1,2,2,2,3,3,3,3,0,0,1,0,1,2,0,1,2,3,0,1,0,2,1,0,3,2,1,0,mode,ψi,1,2,√,3,2,x,√,3,2,y,√,5,4,(3x2,−,1),3,2,xy,√,5,4,(3y2,−,1),√,7,4,(5x3,−,3x),√,15,4,(3x2,−,1)y,√,15,4,x(3y2,−,1),√,7,4,(5y3,−,3y),Table,1:,Table,showing,the,lowest,ten,modes,in,2-D.,The,non-zero,entries,of,the,x-stiffness,matrix,,deﬁned,as,(Sx)ij,≡,(cid:90),1,(cid:90),1,(cid:18),dy,ψi,dx,−1,−1,(cid:19),∂ψj,∂x,are,(Sx)01,=,(Sx)06,=,(Sx)13,=,(Sx)24,=,(Sx)36,=,(Sx)47,=,(Sx)58,=,√,√,√,√,√,√,√,3,7,15,3,35,15,3,(61),(62),(63),The,matrix,is,sparse,(7,non-zero,entries,out,of,a,possible,100),and,the,entries,are,simply,square,roots,of,odd,numbers.,Note,that,the,higher-order,modes,function,effectively,as,ﬁnite-volume,as,there,is,no,coupling,between,them,from,the,stiffness,matrices.,The,edge,mass,matrices,show,a,conspicuous,pattern,that,allows,a,computational,efﬁciency,gain,(Fig.2).,Modes,associated,to,a,particular,polynomial,order,repeat,the,structure,present,at,lower,or-,ders,,up,to,a,multiplier,,while,adding,one,new,row,structure.,The,‘repeated’,rows,summed,against,22,Figure,2:,Repeating,pattern,in,edge,mass,matrix,for,the,top,edge,of,a,quadrilateral,element,where,the,index,i,labels,the,row,and,j,the,column.,Numbers,shown,here,are,the,actual,matrix,entries,multiplied,by,two,and,then,squared,,in,order,to,make,the,underlying,structure,clear;,the,shading,colour,indicates,rows,in,which,the,squared,matrix,components,are,related,to,the,‘ﬁrst’,row,of,a,particular,colour,by,multiplication,by,an,odd,integer.,Note,that,the,tiers,(modes,of,common,polynomial,order),have,columns,where,the,non-zero,components,correspond,to,odd,entries,of,the,1,,3,,5,,...,times,tables.,the,vector,of,ﬁeld,differences,are,computed,simply,by,rescaling,previously-computed,sums.,Possi-,bly,the,identity,dPn(x),dx,=,n,x2−1,(xPn(x),−,Pn−1(x)),relating,the,derivative,of,a,Legendre,polynomial,to,two,Legendre,polynomials,,plays,a,role.,Note,that,in,nodal,DG,,the,numerical,ﬂuxes,involve,only,the,nodes,that,are,localized,at,the,element,boundary;,however,,the,full,mass,matrix,means,involving,all,modes,in,the,ﬂux,calculation.,B.2.1,Concluding,remarks,The,efﬁciencies,shown,here,are,yet,more,pronounced,in,three,dimensions,,as,observed,earlier,in,the,discussion,on,sum-factorization.,If,it,is,necessary,to,include,non-rectilinear,features,in,the,mesh,,it,is,possible,to,mesh,using,a,mixture,of,simplices,near,boundaries,and,rectilinear,elements,in,bulk,regions.,An,example,mesh,is,shown,in,Fig.3.,In,this,case,,the,strategy,design,pattern,could,be,used,to,enable,the,run-time,selection,of,algorithm,to,compute,the,numerical,ﬂux,depending,on,the,types,of,elements,involved,(discussion,of,numerical,stability,questions,associated,to,such,processes,is,omitted,here).,23,Figure,3:,Example,mesh,using,regular,quadrilaterals,in,bulk,regions,and,constrained,Delaunay,triangulation,local,to,non-axis-aligned,geometry,,in,this,case,a,D-shaped,closed,curve,intended,as,proxy,for,a,half-cross-section,of,a,tokamak,ﬁrst,wall.,24

:pdfembed:`src:_static/CD-EXCALIBUR-FMS0067-M6c.1_ExcaliburFusionModellingSystem.pdf, height:1600, width:1100, align:middle`