CD-EXCALIBUR-FMS0031-M2.5.1_SelectMorModelOrderReduction
========================================================

.. meta::
   :description: technical note

   :keywords: ExCALIBUR,Select,techniques,for,MOR,(Model,Order,Reduction),M2.5.1,Abstract,The,report,describes,work,for,ExCALIBUR,project,NEPTUNE,at,Milestone,2.5.1.,This,report,includes,an,overview,of,three,important,classes,of,Model,Order,Reduction,(MOR),approach:,re-,duced,basis,methods,,proper,orthogonal,decomposition,,and,proper,generalized,decomposition.,Also,included,is,a,discussion,of,techniques,for,data,assimilation,,taken,from,the,context,of,nu-,merical,weather,prediction,,and,how,these,techniques,overlap,with,MOR.,Of,particular,interest,for,NEPTUNE,are,methods,capable,of,addressing,nonlinear,problems,and,so,some,attention,is,drawn,to,the,treatment,of,nonlinear,effects,in,numerical,weather,prediction.,The,report,contains,also,some,initial,suggestion,of,how,the,techniques,described,might,be,ap-,plied,in,the,initial,proxyapp,stage,of,NEPTUNE;,however,,a,fuller,discussion,and,more,detailed,conclusions,have,to,be,deferred,until,the,outputs,of,the,defrayed,MOR,grant,have,been,received.,UKAEA,REFERENCE,AND,APPROVAL,SHEET,Client,Reference:,UKAEA,Reference:,CD/EXCALIBUR-FMS/0031,Issue:,Date:,1.00,15,March,2021,Project,Name:,ExCALIBUR,Fusion,Modelling,System,Prepared,By:,Name,and,Department,Ed,Threlfall,Wayne,Arter,Signature,N/A,N/A,Date,15,March,2021,15,March,2021,BD,Reviewed,By:,Rob,Akers,15,March,2021,Advanced,Dept.,Manager,Computing,2,1,Introduction,Modern,computational,models,in,physics,and,engineering,achieve,a,high,level,of,complexity,,with,numbers,of,degrees,of,freedom,(alternately,‘state-space,dimension’),frequently,of,order,a,billion,,or,more.,Despite,this,,many,problems,of,interest,are,still,out,of,reach,for,direct,methods,,either,due,to,the,sheer,number,of,degrees,of,freedom,,or,because,the,problem,involves,widely-varying,time,or,space,lengthscales,,as,occurs,not,infrequently,in,multi-physics,scenarios,where,there,are,unresolved,scales,of,variation.,Problems,of,both,these,types,are,associated,with,the,NEPTUNE,use,case,of,magnetically-conﬁned,fusion,,in,which,a,typical,degree,of,freedom,count,is,1021.,In,order,to,surmount,the,problem,of,an,unmanageably-large,state-space,,it,might,be,asked,whether,some,of,the,degrees,of,freedom,,or,certain,combinations,thereof,,can,be,taken,to,be,in,some,sense,more,important,than,others,,with,the,‘others’,being,ignorable.,Such,an,approach,of,retaining,only,the,dominant,behaviour,of,a,system,is,termed,model,order,reduction,(MOR),,with,the,resulting,model,being,referred,to,as,a,reduced-order,model,(ROM),(synonyms,are,a,surrogate,model,or,an,emulator).,These,may,be,hundreds,to,thousands,of,times,faster,in,execution,than,the,high-,dimensional,model,they,approximate.,The,utility,of,a,much,faster,proxy,is,obvious,for,uncertainty,quantiﬁcation,(UQ),,calibration,of,models,to,large,amounts,of,data,,real-time,control,systems,,and,for,allowing,the,solution,of,coupled,,multiscale,problems.,MOR,is,a,fairly,new,ﬁeld,,though,a,rapidly-developing,one,since,it,stands,on,the,shoulders,of,a,large,body,of,techniques,from,linear,algebra,and,systems,theory.,It,is,an,attempt,to,tackle,(if,not,‘solve’,in,the,exhaustive,sense),today’s,‘intractable’,problems,using,advances,in,algorithm,In,an,era,technology,rather,than,hardware;,working,‘smarter’,,rather,than,necessarily,‘harder’.,where,FLOPS,per,watt,is,a,marketable,ﬁgure,of,merit,,the,energy,efﬁciency,of,such,approaches,might,be,considered,a,bonus,,in,addition,to,the,elegance,of,avoiding,doing,unnecessary,work.,There,is,also,the,sheer,convenience,of,being,able,to,make,progress,using,only,light,computing,platforms,,given,that,a,powerful,computer,means,a,physically,large,computer,,currently,and,for,the,foreseeable,future.,The,focus,here,is,on,algorithmic,approaches,to,MOR,,as,opposed,to,the,more,traditional,ap-,plication,of,physicists’,or,engineers’,domain,knowledge.,The,goal,is,a,systematic,procedure,to,identify,the,dominant,emergent,degrees,of,freedom,from,potentially,highly-nonlinear,coupled,sys-,tems.,Such,an,algorithm,works,by,reducing,the,genericity,of,an,existing,computational,method,e.g.,a,ﬁnite,element,framework,,incorporating,problem-speciﬁc,constraints,and,boundary,condi-,tions;,in,this,way,it,constrains,the,physics,and,determines,approximate,‘modes’,dominating,the,system,response.,Thus,,MOR,works,in,tandem,with,generic,discretization,routines,(the,spectral,/,hp,methods,chosen,for,exploration,in,NEPTUNE,are,an,example),rather,than,supplanting,them:,most,ROM,approaches,tend,to,deal,with,the,post-discretization,system.,Some,analyses,seek,to,reduce,the,size,of,the,set,of,input,parameters,,and,hence,the,burden,associated,with,data,as-,similation,-,this,is,not,speciﬁcally,addressed,here,,though,clearly,UQ,techniques,can,be,used,to,identify,the,most,‘important’,input,parameters,in,terms,of,their,effect,on,the,mean,or,variance,of,the,output,(and,this,UQ,might,itself,make,use,of,a,ROM,for,Monte,Carlo,sampling).,Section,2,of,this,report,is,written,with,the,intent,of,surveying,the,currently,popular,techniques,for,MOR:,reduced,basis,methods,,proper,orthogonal,decomposition,,and,proper,generalized,decom-,position,,though,note,that,it,is,intended,as,an,initial,exposition,in,anticipation,of,further,investiga-,3,tive,work.,The,near-term,future,goal,is,to,focus,on,computational,models,of,the,edge,region,of,a,magnetically-conﬁned,fusion,plasma,,which,tends,to,be,described,using,ﬂuid,dynamics,equations,(an,example,of,state-space,basis,reduction,in,a,fusion,context,,using,Krylov,subspace,methods,,can,be,found,in,[1]),,though,ultimately,it,may,be,useful,to,apply,ROMs,also,to,particle,kinetics,[2],and,gyrokinetics,[3].,It,is,worth,noting,that,there,are,currently,several,open-source,MOR,toolkits,and,modules;,a,list,can,be,found,in,[4].,The,use,of,Gaussian,processes,in,model,order,reduction,is,not,included,here,since,this,work,has,been,assigned,to,a,third,party,,though,the,value,of,the,Gaussian,process,approach,in,facilitating,non-intrusive,UQ,is,to,be,noted.,Another,important,area,of,model,validation,is,the,need,to,incorporate,experimental,measurements,(e.g.,in,order,to,ﬁt,parameters,in,a,model,or,in,order,to,deﬁne,an,inverse,problem,for,an,initial,condition).,The,process,of,including,data,into,models,is,called,data,assimilation,(DA).,There,are,various,analytic,techniques,for,data,assimilation,,most,of,which,start,with,the,assumption,of,Gaus-,sian,errors.,Here,,ROMs,ﬁnd,relevance,in,that,there,is,frequently,a,need,to,integrate,a,complicated,model,(at,least,once),in,order,to,propagate,the,information,derived,from,experimental,measure-,ments,and,their,corresponding,uncertainties,either,forward,or,backward,in,time;,an,effective,ROM,can,make,this,process,a,lot,less,cumbersome.,There,is,also,the,fact,that,DA,techniques,can,be,applied,to,the,Bayesian,calibration,of,ROMs,,for,example,,the,ensemble,Kalman,ﬁlter,can,be,used,as,a,parameter,estimation,framework.,Because,of,this,overlap,,an,overview,of,the,basic,DA,methods,is,included,in,Section,3,of,this,report.,This,section,is,really,structured,as,an,exposition,of,analytic,methods,derived,for,the,spe-,cial,case,of,linear,models,and,Gaussian,errors,,generalized,to,treatments,of,real,models,,and,some,mention,of,the,methods,for,incorporating,known,physics,into,the,process,is,made.,The,canonical,example,of,DA,,and,to,a,large,extent,the,context,of,the,discussion,presented,here,,is,numerical,weather,prediction,(NWP);,here,,a,large,amount,of,observational,data,is,available,(in,the,form,of,wind,velocity,,pressure,,humidity),and,the,system,to,be,modeled,,though,based,on,well-understood,microscopic,physics,,comprises,vastly,many,degrees,of,freedom,interacting,non-,linearly,over,a,vast,range,of,lengthscales.,It,goes,without,saying,that,the,ﬁeld,of,meteorology,has,a,great,deal,of,experience,with,DA,(and,to,a,lesser,extent,,MOR,,though,the,entire,workings,of,NWP,could,be,viewed,as,a,MOR),,with,forecasts,having,been,made,on,a,daily,basis,using,meth-,ods,improved,incrementally,over,many,decades.,The,techniques,here,for,extending,the,analytic,Gaussian,results,to,a,real,nonlinear,system,are,of,great,interest,because,one,aim,of,NEPTUNE,is,to,use,knowledge,of,plasma,physics,to,improve,in,a,similar,fashion,our,understanding,of,the,‘weather’,inside,a,tokamak.,Tokamak,physics,also,contains,noteworthy,similarities,to,certain,phe-,nomena,encountered,in,numerical,weather,prediction,,for,example,multiple,scales,and,turbulent,ﬂow.,An,appendix,gathers,some,useful,facts,concerning,Gaussian,statistics.,4,2,Techniques,for,Model,Order,Reduction,For,illustrative,purposes,,a,concrete,deﬁnition,of,a,typical,target,problem,is,given.,In,the,further,interest,of,clarity,,a,linear,time-invariant,system,is,considered,(note,that,nonlinear,models,are,typically,time-updated,using,a,linearization,,though,in,that,case,the,property,of,time-invariance,is,lost).,Consider,an,input,u(t),∈,Rm,to,a,model,with,state,deﬁned,by,x(t),∈,Rn,and,giving,an,output,y(t),∈,Rp:,˙x(t),=,Ax(t),+,Bu(t),y(t),=,Cx(t).,(1),(2),Here,the,number,of,model,states,,that,is,the,dimension,of,the,vector,x(t),,given,by,n,above,,may,be,very,large,(and,generally,is,much,larger,than,the,number,of,inputs,m,and,the,number,of,outputs,p);,in,a,modern,application,n,might,be,taken,to,be,of,the,order,of,a,billion,(with,numbers,such,as,this,,it,is,feasible,to,store,the,vector,of,model,states,but,not,to,store,the,n,×,n,matrix,A).,This,full,model,is,a,high-dimensional,state,space,linear,model,(sometimes,called,the,high-dimensional,model,in,the,context,of,MOR).,Now,MOR,aproaches,aim,to,replace,the,full,internal,state,by,a,smaller,state,vector,,of,size,r,(cid:28),n,,deﬁned,by,x(t),=,V,˜x(t);,then,choosing,another,matrix,deﬁned,by,W,T,V,=,I,the,system,is,˙˜x(t),=,W,T,AV,˜x(t),+,W,T,Bu(t),y(t),=,CV,˜x(t).,(3),(4),Here,W,and,V,are,n,×,r,matrices,(of,tall,,thin,shape).,The,task,of,MOR,approaches,is,to,ﬁnd,the,projection,matrices,W,,V,such,that,the,response,of,this,reduced,system,mimics,as,closely,as,possible,that,of,the,full,system.,The,following,discussions,are,necessarily,brief,and,many,important,considerations,,such,as,the,stability,of,the,ROMs,,are,not,included.,More,detailed,presentations,of,the,material,can,be,found,in,e.g.,the,review,[5],and,also,in,the,documentation,for,the,open-source,Python,MOR,toolkit,pyMOR,[6].,2.1,Reduced,basis,methods,To,focus,on,the,case,of,reduction,for,parameterized,PDEs,,consider,a,family,of,PDEs,deﬁned,by,a,single,equation,with,a,dependence,on,parameters,µ,belonging,to,a,set,P,(this,would,represent,the,inputs,to,the,model).,To,give,a,parabolic,example,,let,the,solution,be,labelled,by,uµ(t);,the,semi-discrete,problem,is,then,,in,weak,form,5,uµ(0),=,u0,(cid:104)v,,˙uµ(t)(cid:105),+,bµ,(v,,uµ(t)),=,f,(v),∀v,∈,Vh,yµ(t),=,g(uµ(t)).,(5),(6),(7),This,is,the,full-order,model,(note,the,ﬁrst,equation,is,simply,the,initial,condition).,States,uµ,be-,long,to,a,large-dimensional,vector,space,Vh,which,spans,the,states,available,to,the,system,post-,discretization.,Here,the,discretized,spatial,part,of,the,PDE,is,contained,in,the,matrix,operator,bµ,,which,also,contains,dependence,on,the,set,of,parameters,(an,example,would,be,a,spatially-,varying,diffusion,constant).,To,seek,an,approximate,solution,on,a,subspace,VN,⊂,Vh,,a,solution,the,same,problem,is,sought,for,vectors,in,the,reduced,set,and,using,an,initial,condition,Galerkin,projected,onto,this,space.,The,general,idea,here,is,to,construct,iteratively,a,reduced,space,for,the,internal,degrees,of,freedom,,given,a,predeﬁned,set,of,parameters,restricted,to,the,region,of,interest,in,parameter,space,and,referred,to,as,the,training,parameter,set.,The,choice,of,vector,to,add,at,a,given,stage,in,the,process,is,the,main,problem,,and,this,choice,is,motivated,by,the,deﬁnition,of,the,Kolmogorov,N,-width,dN,:,dN,=,inf,VN,⊆V,dim(VN,)≤N,sup,µ∈P,inf,v∈VN,(cid:107)u(µ),−,v(cid:107).,(8),the,This,quantity,,which,gives,an,upper,bound,for,the,possible,quality,of,a,reduced,space,i.e.,worst-case,error,,needs,some,explanation.,For,a,particular,solution,u(µ),of,the,full-order,model,(known,as,a,snapshot),,the,best,approximation,in,the,current,reduced,space,is,ﬁrst,computed,(this,is,the,inner,inﬁmum,,which,simply,means,identifying,the,reduced-space,vector,v,that,best,approximates,the,snapshot),,then,the,error,in,this,is,maximized,(the,supremum),over,the,training,set,of,µ,,giving,the,worst-case,best,approximation.,Taking,the,inﬁmum,of,this,over,all,possible,reduced,spaces,VN,gives,the,Kolmogorov,N,-width.,In,practice,,the,inner,inﬁmum,is,dealt,with,by,orthogonal,projection,of,the,snapshot,u(µ),into,VN,.,Denoting,the,reduced,basis,vectors,by,vi,for,i,=,1,,...,,N,the,relevant,coefﬁcents,(projections),λi,can,be,found,by,solving,N,(cid:88),j=1,λj(cid:104)vj,,vi(cid:105),=,(cid:104)v,,vi(cid:105),,(9),which,can,also,be,expressed,as,the,matrix,equation,GΛ,=,R;,the,matrix,G,is,referred,to,as,the,Gramian.,Note,that,a,naive,implementation,would,lead,to,the,condition,number,of,the,Gramian,getting,increasingly,large,as,new,vectors,are,added,into,the,reduced,basis,(because,of,the,in-,creasing,amount,of,linear,dependence);,in,order,to,avoid,this,issue,,the,reduced,basis,tends,to,be,orthonormalized,using,the,standard,Gram-Schmidt,procedure,(reducing,the,Gramian,to,an,identity,matrix).,Note,the,basis,construction,is,an,example,of,the,method,of,snapshots,due,to,the,use,of,particular,solutions,of,the,full-order,model.,6,Each,new,vector,added,to,the,reduced,basis,ought,to,give,in,some,sense,the,maximum,possible,improvement,in,the,approximation,and,this,can,be,achieved,by,so-called,‘greedy’,methods:,the,aim,is,to,ﬁnd,the,parameter,value,µ∗,that,maximizes,the,error,between,the,reduced,model,and,the,full-order,model,,then,add,the,solution,vector,corresponding,to,this,µ∗,to,the,reduced,basis,(having,ﬁrst,subtracted,the,part,projected,onto,the,existing,reduced,state,space).,This,search,over,the,(remaining),parameters,in,the,training,set,forms,the,inner,loop,of,the,algorithm;,clearly,,using,the,full-order,model,here,(giving,the,strong,greedy,algorithm),incurs,a,signiﬁcant,speed,cost,in,constructing,the,ROM.,The,process,can,be,streamlined,by,using,the,current,ROM,as,a,proxy,for,the,full-order,model,in,this,inner,loop,(giving,the,weak,greedy,algorithm),and,effectively,increasing,the,size,of,the,training,sets,(and,thereby,the,dimension,of,the,parameter,spaces),that,can,be,used.,A,further,simpliﬁcation,,in,order,to,avoid,needing,to,recompute,the,system,matrices,for,each,choice,of,parameters,,involving,repeated,computation,of,the,relevant,bilinear,forms,using,the,full-order,model,,is,to,assume,that,the,parameter,dependence,of,the,operators,can,be,put,into,separable,form,(also,known,as,an,afﬁne,decomposition),,viz.,bµ(v,,u),=,Q,(cid:88),q=1,θq(µ),bq(v,,u).,(10),This,gives,what,is,termed,online,efﬁciency,-,new,parameter,values,can,be,tested,without,expensive,recomputation.,Clearly,this,will,not,work,in,the,case,of,a,nonlinear,operator,as,the,linearized,operators,are,then,state-dependent.,One,solution,to,this,problem,is,empirical,interpolation:,the,behaviour,of,the,nonlinear,operator,is,represented,by,a,small,number,of,pointwise,evaluations,with,an,(afﬁne),interpolation,to,general,parameter,values.,The,technique,can,also,be,applied,to,problems,where,there,is,no,afﬁne,decomposition,of,the,operators.,2.2,Proper,orthogonal,decomposition,(POD),POD,is,another,method,for,creating,a,reduced,basis,out,of,snapshot,data,and,it,is,related,to,the,familiar,Principal,Components,Analysis,(PCA),used,in,statistical,analysis,[7],and,also,to,the,Karhunen-Lo,´eve,expansion,used,in,stochastic,process,modelling.,In,PCA,,a,vector,of,random,variables,x,is,subjected,to,a,linear,transformation,to,a,new,variable,y,such,that,the,covariance,matrix,(cid:104)(y,−,¯y)(y,−,¯y)T,(cid:105),of,the,new,variable,is,diagonal,(this,can,always,be,done,,since,the,covari-,ances,form,a,real,symmetric,matrix).,The,Karhunen-Lo,´eve,expansion,is,basically,the,continuous,analogue,of,this.,PCA-type,analyses,rely,on,the,basic,fact,that,a,matrix,acting,on,a,general,vec-,tor,will,give,greater,weight,to,components,of,the,vector,that,lie,in,the,subspace,spanned,by,the,eigenvectors,of,the,matrix,with,the,largest,eigenvalues.,In,POD,the,snapshot,vectors,are,assembled,into,the,snapshot,matrix,(this,is,a,tall,,thin,matrix:,the,dimension,of,the,full,state,is,n,and,the,number,of,samples,ns,,with,ns,(cid:28),n),A,=,u(µ1),u(µ2),...,...,,,,,...,...,7,·,·,·,·,·,·,u(µns),·,·,·,...,...,,,,,(11),Then,the,singular,value,decomposition,(SVD),of,A,is,computed,A,=,U,ΣV,T,.,(12),For,clarity,,here,U,is,n,×,ns,and,the,columns,of,this,matrix,are,the,left-singular,vectors,of,A;,V,is,ns,×,ns,and,the,columns,are,the,right-singular,vectors,of,A.,The,diagonal,entries,of,the,ns,×,ns,matrix,Σ,are,the,POD,singular,values,of,A,and,are,arranged,in,monotonically,decreasing,order,(denoting,the,components,of,Σ,as,σ1,≥,σ2,...,≥,σns).,The,magnitude,of,each,of,these,singular,values,(which,can,be,clearly,seen,to,generalize,the,eigenvalues,for,a,square,matrix),tells,how,important,is,each,vector,in,the,columns,of,U,for,the,approximation,of,the,snapshot,data.,The,procedude,is,then,to,retain,a,given,number,of,these,vectors,,and,in,the,sense,of,minimizing,the,L2-norm,,an,optimal,approximation,space,is,obtained.,It,is,worth,noting,that,individual,errors,may,be,large,,despite,the,least-squares,optimality,(this,is,different,to,the,greedy,methods,applied,in,the,preceding,subsection).,The,POD,is,an,efﬁcient,lower-dimensional,representation,of,the,snapshot,data,in,the,sense,that,if,the,ﬁrst,r,singular,vectors,are,retained,r,≤,ns,then,the,POD,basis,minimizes,the,last-squares,error,relative,to,the,ns-dimensional,reduced,space;,and,this,least-squares,error,is,represented,by,the,sum,of,the,squares,of,the,remaining,eigenvalues,,a,fact,that,can,be,used,to,choose,an,appropriate,size,of,POD,basis:,one,approach,,referred,to,as,the,fractional,‘energy’,in,the,POD,modes,,is,to,choose,r,such,that,(cid:80)r,(cid:80)ns,i=1,σ2,i,i=1,σ2,i,>,κ,(13),with,κ,being,something,approaching,unity,,e.g.,99.9%.,Note,that,this,error,assessment,applies,only,to,the,error,in,the,reconstruction,of,the,snapshots,and,not,the,error,relative,to,the,full,model.,Error,analysis,with,this,wider,scope,is,not,possible,in,all,cases,-,see,for,example,[5],,p.9.,There,is,also,the,issue,of,how,best,to,choose,the,snapshot,set,(optimal,snapshot,selection).,Uniform,sampling,is,likely,to,be,effective,only,for,problems,with,a,handful,of,parameters;,for,larger-,dimensional,parameter,spaces,,sparse,grid,sampling,or,greedy,sampling,may,be,used.,Again,,UQ,analysis,to,exclude,less,important,parameters,and,restriction,to,physically,reasonable,location,in,parameter,space,are,useful.,Note,that,because,the,POD,basis,is,constructed,from,sampled,solutions,,this,method,can,be,applied,straightforwardly,to,linear,and,nonlinear,systems.,2.3,Proper,generalized,decomposition,(PGD),Many,MOR,techniques,(including,the,POD,method,outlined,above),operate,in,a,sequential,fashion,,constructing,a,reduced,basis,and,then,projecting,the,problem,solution,onto,it.,A,potential,issue,here,is,that,if,the,initial,reduced,basis,is,in,some,way,unsuitable,,the,second,step,will,yield,a,poor,reduced,model,,and,the,only,way,to,proceed,is,to,repeat,step,one,and,construct,a,new,basis.,One,8,way,to,address,this,problem,is,to,seek,a,method,that,works,incrementally,i.e.,the,reduced,basis,is,capable,of,being,enriched,with,new,states,during,the,construction,of,the,ROM.,In,PGD,the,simpliﬁcation,is,made,that,the,solution,to,the,problem,is,separable,in,time,and,in,each,(or,some),of,the,space,coordinates.,Thus,the,solution,to,a,parametric,problem,may,be,sought,in,the,form,uN,(x,,t,,µ1,,...µM,),≈,N,(cid:88),i=1,Xi(x)Ti(t),M,(cid:89),m=1,P,m,i,(pm).,(14),The,important,question,now,is,the,algorithm,for,constructing,the,separated,functions.,An,enrich-,ment,scheme,is,used,,with,e.g.,un(x,,t),=,un−1(x,,t),+,Xn(x)Tn(t),(15),being,substituted,into,the,equation,and,the,nonlinear,problem,for,the,unknown,functions,Xn(x),and,Tn(t),solved.,The,enrichment,scheme,itself,is,an,iterative,process,that,involves,solving,for,the,sequences,X,0,n,,,where,e.g.,Xn,is,taken,to,be,X,p,n,once,a,tolerance,n,and,T,0,n,,...,,X,p,n,,,...,,T,p,n,,X,1,n,,,T,1,(cid:107)X,p,n(x)T,p,n,(t),−,X,p−1,n,(x)T,p−1,n,(t)(cid:107),<,(cid:15),(16),is,reached,(here,(cid:107)...(cid:107),is,a,suitable,norm).,The,procedure,involves,an,initial,guess,for,all,but,one,of,the,unknown,functions,in,the,product,and,then,solving,the,equation,for,the,remaining,unknown,(this,is,called,an,alternating,direction,strategy).,The,actual,solution,mechanism,is,typically,a,greedy,algorithm,applied,to,the,weak,formulation,of,the,problem.,It,is,well,worth,noting,that,the,extent,to,which,the,spatial,dependence,can,be,separated,stems,from,the,choice,of,spatial,computational,mesh:,meshes,with,symmetry,directions,result,in,a,reduction,of,the,complexity,of,the,problem,needed,to,solve,each,iteration,of,the,alternating,direction,problem.,As,an,example,,a,mesh,of,rectilinearly-aligned,hexahedra,would,represent,the,easiest,case,(three,spatial,symmetry,directions),,with,an,extruded,(unstructured),triangular,mesh,of,prisms,being,easier,than,the,fully,three-dimensional,tetrahedral-meshed,case.,This,is,a,clear,example,of,the,relevance,of,the,spatial,discretization,to,the,efﬁciency,of,subsequent,algorithms.,Note,that,the,modes,obtained,using,PGD,are,not,necessarily,orthogonal,,unlike,POD,modes.,2.4,System-theoretic,approaches,The,idea,behind,balanced,truncation,is,to,intelligently,reduce,the,state,vector,by,removing,states,that,are,either,hard,to,reach,or,hard,to,observe.,This,is,done,by,computing,the,reachability,Gramian,P,and,the,observability,Gramian,Q,deﬁned,by,AP,+,PAT,+,BBT,=,0,AT,Q,+,QA,+,CT,C,=,0;,9,(17),(18),note,the,matrices,A,,B,,C,are,as,deﬁned,in,2,above.,To,ﬁnd,the,states,to,remove,,a,balancing,transformation,that,simultaneously,diagonalizes,P,and,Q,is,sought;,and,matrices,V,and,W,(again,,deﬁned,in,2),are,computed.,There,are,also,frequency-domain,approaches,,for,example,,rational,interpolation,involves,approx-,imating,the,transfer,function,(the,Laplace,transform,of,the,impulse,response),of,the,system,by,a,rational,function.,More,information,can,be,found,at,[6].,3,Data,Assimilation,The,two,main,classes,of,data,assimilation,method,are,sequential,methods,,used,to,propagate,the,state,of,the,model,and,the,information,from,observations,forward,in,time,one,step,at,a,time,(em-,bodied,in,the,Kalman,ﬁlter);,and,variational,methods,,in,which,the,assimilation,is,performed,for,all,timesteps,at,once,(hence,,the,information,contained,in,an,observation,at,a,given,time,propagates,backward,and,forward,in,time).,Both,classes,of,method,are,based,on,analytic,results,derived,for,the,combination,of,Gaussian-distributed,errors,and,using,linear,dynamical,models.,3.1,Kalman,ﬁltering,The,ﬁltering,problem,is,how,to,maintain,an,optimally-predictive,stochastic,model,in,the,presence,of,two,sources,of,error:,i),model,error,(also,known,as,system,noise);,and,ii),observation,error.,This,means,calculating,the,PDF,for,the,state,at,a,given,point,in,time,,conditioned,on,observations,up,to,that,time.,First,,the,model,state,is,propagated,for,one,time,step,into,the,future,and,this,propagation,incurs,an,error,known,as,the,system,noise.,The,propagation,dynamics,may,be,linear,or,not,,though,linearity,is,required,in,order,to,keep,the,Gaussian,statistics,for,which,the,Kalman,ﬁlter,is,strictly,optimal.,Secondly,,new,experimental,observations,are,assumed,to,become,known,at,the,new,time,,in,the,form,of,measurement,data,with,its,own,errors,,called,observation,error.,There,are,therefore,two,fundamental,steps:,combining,the,error,in,the,model,at,the,previous,timestep,with,the,model,error,incurred,by,the,time-advance,,then,conditioning,the,resulting,system,on,the,probability,distribution,associated,to,the,new,observation.,The,problem,of,propagating,sequentially,the,probability,density,function,(PDF),p(x),of,the,state,from,time,index,i,to,i,+,1,is,easily,understood,,at,least,in,the,case,of,a,linear,system,(and,it,might,be,noted,here,that,many,models,are,propagated,in,time,using,a,linearization,about,their,true,dynamics).,A,simple,pedagogical,example,is,that,of,a,random,walk,where,the,individual,(spatial),‘step’,is,itself,a,random,variable,,(cid:15)i,,with,PDF,qi(x),for,each,individual,step,i;,thus,,the,state,of,the,walk,updates,as,xi+1,=,xi,+,(cid:15)i,(19),and,the,evolution,of,the,PDF,is,given,by,the,convolution,10,pi+1(x),=,(cid:90),+∞,−∞,pi(x,−,u)qi(u)du.,(20),This,formula,,which,can,be,interpreted,as,a,probabilistic,sum,over,all,possible,update,steps,,is,most,easily,solved,by,transforming,to,the,Fourier,domain,,in,which,the,convolution,becomes,a,simple,product,of,the,Fourier-transformed,PDFs,(known,as,characteristic,functions):,viz.,,indicat-,ing,a,Fourier-transformed,quantity,with,a,tilde,,˜pi+1(k),=,˜pi(k)˜qi(k).,(21),This,can,be,used,to,show,-,basically,by,completing,the,square,-,that,,if,the,initial,p0(x),is,Gaussian,and,the,distribution,of,the,step,lengths,is,also,Gaussian,,then,the,error,in,the,sum,is,always,Gaussian,and,that,the,variance,is,incremented,by,the,variance,of,the,individual,steps,(the,familiar,square-root,dependence,of,the,expected,displacement,via,a,random,walk,can,be,seen,to,emerge,from,this,rule).,Here,it,is,assumed,that,the,errors,in,different,steps,labelled,by,i,are,uncorrelated.,Note,this,works,because,of,the,basis,change,to,the,eigenspace,of,the,space,translation,operator,(this,sort,of,diagonalization,-,a,transform,to,an,‘uncorrelated’,basis,-,comes,up,again,and,again,e.g.,in,Principal,Components,Analysis,,mentioned,in,Subsection,2.2).,These,simple,conclusions,survive,for,any,choice,of,model,time-update,that,is,linear.,The,other,thing,to,consider,is,the,fact,that,,as,well,as,the,system,noise,(represented,by,the,distri-,bution,qi(x),in,the,above),,there,are,also,observational,data,that,take,the,form,of,measurements,,these,with,their,own,errors,(observation,error).,Given,the,PDF,of,the,state,at,time,index,i,+,1,(i.e.,pi+1(x),in,the,above),and,the,new,observation,,the,question,of,the,correct,new,PDF,for,the,system,is,answered,by,Bayes’,theorem,,which,enables,us,to,make,use,of,the,information,about,the,state,from,the,previous,time,step,(in,fact,,recursively,,all,past,time,steps,and,prior,observations),and,the,new,information,in,the,current,observation.,Bayes’,theorem,is,usually,stated,as,P,(A|B),=,P,(B|A)P,(A),P,(B),,,(22),where,P,(A|B),is,to,be,read,as,the,probability,distribution,of,A,given,prior,knowledge,B.,The,term,P,(A),represents,prior,knowledge,and,is,simply,referred,to,as,the,prior;,the,left-hand,distribution,,incorporating,this,knowledge,,is,called,the,posterior,distribution.,To,continue,in,the,vein,of,treating,the,simplest,possible,example,,consider,the,following,Bayesian,derivation,of,the,posterior,,given,a,prior,p(x),∝,exp,(cid:0)−(x,−,x0)2/2σ2,(cid:1),and,an,observation,xe,taken,from,a,distribution,∝,exp,(cid:0)−(x,−,xe)2/2σ2,(cid:1).,Up,to,normalization,,the,posterior,distribution,is,simply,the,product,of,these,two,PDFs,,giving,a,new,Gaussian,with,a,variance,σ,that,is,clearly,given,by,0,e,1,σ2,=,1,σ2,0,+,1,σ2,e,,,11,(23),the,new,mean,given,by,µ,=,σ2,;,both,of,these,formulae,can,be,obtained,by,minimizing,the,variance,when,combining,two,uncorrelated,random,variables,-,a,simple,example,of,least,squares.,The,beneﬁt,of,all,this,is,that,observational,‘evidence’,is,weighted,according,to,its,degree,of,uncer-,tainty.,e,x0+σ2,0xe,σ2,0+σ2,e,It,is,somewhat,amusing,that,this,update,involves,a,square,completion,in,real,space,while,the,update,for,the,model,error,had,a,square,completion,in,k-space,(i.e.,adding,Gaussian,random,variables,involves,adding,real-space,variances,,while,conditioning,involves,adding,k-space,vari-,ances,-,hence,the,reciprocals,in,23).,This,example,demonstrates,the,property,that,a,Gaussian,prior,and,a,Gaussian-distributed,observation,error,lead,to,a,Gaussian,posterior,(and,note,that,the,variance,in,the,posterior,does,not,depend,on,the,actual,measured,value).,It,is,also,clear,that,the,model,expectation,value,undergoes,a,discontinuous,jump,on,incorporation,of,the,new,observation.,Again,,the,basic,conclusions,here,survive,for,any,observation,y(x),that,is,a,linear,function,of,the,underlying,model,variable,x.,Taken,together,,and,extended,to,a,general,linear,system,of,random,variables,(the,scalar,quantities,above,become,vectors;,the,variances,become,convariance,matrices),,the,two,key,ideas,above,(adding,Gaussian,random,variables,and,Gaussian-on-Gaussian,conditioning),combine,to,give,an,analytic,ﬁltering,algorithm,known,as,the,Kalman,ﬁlter.,Taking,the,system,dynamics,to,be,and,the,observation,deﬁnition,to,be,xk+1,=,M,xk,+,ηk,yk,=,Hxk,+,(cid:15)k,(24),(25),and,the,system,noise,and,observation,errors,respectively,ηk,∼,N,(0,,Q),and,(cid:15)k,∼,N,(0,,R),,there,follow,formulae,for,the,expectation,vector,and,covariance,matrix,of,the,model,at,time,step,k,,called,the,analysis,values,and,given,subscript,a,(and,because,of,the,Gaussian,statistics,,this,exhausts,the,properties,of,the,model):,(cid:18),k,=,P,a,xa,k,H,T,R−1yo,k,+,(cid:17)−1,(cid:16),P,f,k,(cid:19),,,xf,k,P,a,k,=,(cid:18)(cid:16),P,f,k,(cid:17)−1,+,H,T,R−1H,(cid:19)−1,,,where,the,forecast,values,for,the,mean,and,covariance,are,and,f,=,M,xa,xk,k−1,12,(26),(27),(28),P,f,k,=,M,P,a,k−1M,T,+,Q.,(29),These,forms,make,manifest,the,similarity,to,the,scalar,Bayesian,formulae,above;,there,are,alter-,nate,forms,which,may,or,may,not,be,easier,to,evaluate,in,practice,due,to,the,different,dimensions,of,the,matrices,to,be,inverted.,Note,that,the,analysis,covariance,is,updated,using,a,procedure,similar,to,23,above;,the,covariance,associated,to,the,measurement,error,is,added,in,reciprocal.,One,alternative,form,for,the,update,of,the,mean,is,k,=,xf,xa,k,+,P,a,k,H,T,R−1,(cid:16),k,−,Hxf,yo,k,(cid:17),;,(30),the,matrix,K,≡,P,a,is,known,as,the,Kalman,gain,matrix,and,the,observation-forecast,difference,the,innovation.,(The,second,form,of,K,can,be,derived,from,k,H,T,+,R,k,H,T,R−1,=,P,f,HP,f,k,H,T,(cid:16),(cid:17)−1,the,ﬁrst,by,postmultiplying,by,the,identity,in,the,form,the,formula,connecting,the,forecast,and,analysis,variances.),(cid:16),HP,f,k,H,T,+,R,HP,f,k,H,T,+,R,(cid:17),(cid:16),(cid:17)−1,and,using,The,Kalman,ﬁlter,is,optimal,(indeed,analytic),in,the,case,of,strictly,Gaussian,errors,,a,property,that,is,destroyed,if,either,the,system,dynamics,or,the,measurements,involve,nonlinearity.,In,most,practical,applications,the,dynamics,are,nonlinear,,and,observation,errors,are,often,manifestly,non-Gaussian,(for,examples,such,as,humidity,,the,values,are,strictly,positive,and,bounded,from,above,by,physics,e.g.,saturation).,The,absence,of,Gaussianity,means,that,the,conditional,mean,and,conditional,modes,differ;,also,,properties,such,as,the,new,variance,being,independent,of,the,measured,values,and,the,fact,that,the,incorporation,of,the,observation,decreases,the,variance,may,not,hold.,Nonlinear,systems,are,treated,using,the,extended,Kalman,ﬁlter,,in,which,the,mean,propagates,according,to,the,nonlinear,dynamics,and,the,variance,is,evolved,according,to,a,linearization,about,this,mean,(see,the,next,subsection,for,more,on,this).,In,practice,,it,is,difﬁcult,to,apply,the,Kalman,ﬁlter,directly,to,a,high-dimensional,system,because,of,the,need,to,store,the,covariance,matrices,of,large,numbers,of,variables,(typically,106,and,up).,One,possible,approach,is,to,replace,the,continuous,PDF,and,the,full,covariance,matrix,with,a,set,of,random,samples,from,the,appropriate,distributions,,i.e.,resort,to,Monte,Carlo.,For,such,a,sample,,the,Kalman,ﬁlter,is,approximated,by,the,ensemble,Kalman,ﬁlter,(EnKF),and,,again,,the,easiest,case,to,understand,is,the,linear,Gaussian,scenario,,under,which,there,is,a,simple,procedure,for,generating,random,variables,taken,from,the,correct,analysis,distribution.,Starting,with,xi,drawn,from,the,forecast,distribution,(noting,that,here,i,denotes,one,sample,in,the,ensemble),and,observation,error,(cid:15)i,taken,from,N,(0,,R),,form,the,combination,ξi,=,xi,+,K,(cid:0)yo,−,(Hxi,+,(cid:15)i)(cid:1),.,(31),It,is,also,necessary,to,approximate,the,Kalman,gain,matrix,K,and,the,appropriate,procedure,is,to,form,the,approximate,gain,ˆK,=,XY,T,(Y,Y,T,)−1,,(32),13,where,X,is,a,matrix,whose,columns,are,(Ne,−,1)−1,(cid:0)xi,−,ˆxf,(cid:1),and,Y,is,a,matrix,with,columns,(Ne,−,1)−1,(cid:0)Hxi,+,(cid:15)i,−,H,ˆxf,−,ˆ(cid:15)(cid:1),(here,Ne,is,the,size,of,the,ensemble,and,in,the,limit,Ne,→,∞,the,mean,and,covariance,converge,to,those,given,by,the,Kalman,ﬁlter,update).,There,are,many,other,details,but,the,key,point,is,that,the,Monte,Carlo,approach,avoids,the,curse,It,is,clear,that,ensemble-based,of,dimensionality,for,very,large,systems,of,correlated,variables.,schemes,involve,potentially,a,large,number,of,model,evaluations,,implying,that,an,efﬁcient,ROM,could,be,of,great,utility,here.,Note,that,the,EnKF,can,also,be,used,for,parameter,estimation;,in,joint,estimation,,the,parameters,are,included,as,additional,degrees,of,freedom,in,the,state,(there,are,also,other,applications,of,extending,the,state,beyond,the,degrees,of,freedom,of,the,numerical,model).,Such,an,approach,can,be,used,to,calibrate,a,reduced,order,model;,the,advantage,of,doing,this,,as,opposed,to,an,elementary,least-squares,ﬁt,,is,that,the,calibration,is,done,‘correctly’,in,a,Bayesian,sense.,For,a,ROM,with,r,internal,states,and,p,model,parameters,and,assuming,Ny,measurements,,the,approximate,gain,(32),is,a,matrix,of,dimensions,(r,+,p),×,Ny,,which,is,relatively,inexpensive,to,compute,and,use;,also,note,that,there,are,also,techniques,to,avoid,needing,to,compute,the,Ny×Ny,matrix,inverse,(cid:0)Y,Y,T,(cid:1)−1,.,A,more,detailed,introduction,to,the,Kalman,ﬁlter,can,be,found,in,[8].,3.2,Four-dimensional,variational,data,assimilation,and,methods,from,weather,pre-,diction,Four-dimensional,variational,data,assimilation,or,4D-Var,is,a,procedure,that,minimizes,the,mis-,match,between,a,temporal,sequence,of,model,states,and,a,set,of,observational,data.,Importantly,,in,doing,so,it,propagates,information,forward,and,backward,in,time,(the,Kalman,ﬁlter,propagates,information,forward,only).,Note,that,the,problem,of,ﬁtting,a,model,to,a,set,of,measurements,all,at,a,particular,point,in,time,is,termed,3D-Var.,What,is,referred,to,as,weak-constraint,4D-Var,allows,for,uncertainty,in,the,dynamical,propagation,(i.e.,system,noise),and,works,by,minimizing,a,scalar,functional,such,as,J(x0,,x1,,...,,xK),=,1,2,(x0−xb,0)T,(cid:16),P,b,0,(cid:17)−1,(x0−xb,0)+,1,2,K,(cid:88),(Hkxk−yk)T,(Rk)−1,(Hkxk−yk)+,k=0,1,2,K−1,(cid:88),k=0,(xk+1−Mkxk)T,(Qk)−1,(xk+1−Mkxk),(33),The,terms,here,represent,,respectively,,the,deviation,of,the,ﬁtted,model,from,the,initial,back-,ground,data,,the,error,in,the,observed,quantities,,and,the,model,error.,The,assimilation,is,done,over,a,sequence,of,K,+,1,points,in,time,labelled,by,the,index,k.,It,is,clear,,at,least,in,the,lin-,ear,,Gaussian-error,case,,that,J(x0,,x1,,...,,xK),represents,the,negative,logarithm,of,the,PDF,and,hence,its,minimization,gives,a,maximum-likelihood,estimate,for,the,system,state,conditioned,on,the,observed,data.,In,the,case,of,a,realistic,scenario,,the,problem,with,this,is,that,a,direct,minimization,would,require,the,evaluation,of,a,large,number,of,derivatives,,each,of,which,would,involve,a,speciﬁc,integration,14,of,the,dynamical,model.,Making,the,method,feasible,is,the,adjoint,method,,a,technique,that,allows,the,numerical,computation,of,the,gradient,of,a,scalar,function,at,a,cost,of,at,most,a,few,times,the,cost,of,the,direct,computation,of,that,function.,The,adjoint,method,is,basically,a,recursive,trick,for,calculating,the,gradients,of,a,function,of,the,form,f,(x),=,1,2,K,(cid:88),k=0,(xk,−,yk)T,C−1,(xk,−,yk),(34),in,the,particular,case,that,xk+1,=,Mkxk.,The,price,of,using,this,technique,is,that,an,adjoint,model,must,also,be,coded,,in,addition,to,the,actual,model.,In,the,case,of,a,linear,model,and,linear,measurements,,the,problem,is,amenable,to,the,analytic,,Gaussian,techniques,outlined,in,the,previous,subsection.,However,,even,if,the,model,error,is,ignored,(giving,what,is,called,deterministic,4D-Var),,it,turns,out,that,there,are,severe,problems,ﬁtting,these,models,to,observations,for,the,case,of,real,weather,models,,which,are,not,linear.,As,so,far,deﬁned,,a,vector,x,is,sought,that,minimizes,the,4D-Var,penalty,function,J(x),=,1,2,K,(cid:88),k=0,(cid:16),x,−,xb(cid:17)T,B−1,(cid:16),x,−,xb(cid:17),+,(cid:88),1,2,k,=,0K,(y,−,yo)T,R−1,(y,−,yo),,,(35),where,the,ﬁrst,quadratic,term,is,a,Gaussian,background,prior.,This,extremization,is,done,using,local,gradients,evaluated,using,the,adjoint,method.,The,problem,is,that,this,gradient,typically,fails,to,lead,to,a,minimum,of,the,penalty,function,,which,is,nonlinear,due,to,the,nonlinearity,of,y,=,H(M,(x)).,Examples,of,atmospheric,physics,which,lead,to,this,failure,are,given,in,[9];,these,examples,split,into,fast,processes,not,captured,by,scales,simulated,in,the,model,and,‘macroscopic’,phenomena,(e.g.,cloud,formation),not,captured,by,local,gradients.,An,overall,issue,seems,to,be,attempts,to,predict,large-scale,nonlinear,phenomena,based,on,local,gradients,(equivalently,,the,forecasting,model,that,has,been,written,down,is,chaotic).,Note,that,this,problem,relates,to,the,evolution,of,the,prior,under,the,model,and,the,observations,neglecting,any,additional,system,noise,or,measurement,error.,Another,issue,is,that,outside,the,linear,,Gaussian,case,,minimizing,the,argument,of,the,negative,exponential,does,not,estimate,the,mean,of,the,distribution,-,it,estimates,the,mode.,A,strategy,for,dealing,with,this,is,to,track,the,evolution,of,the,mean,using,the,full,nonlinear,model,¯M,and,approximate,the,evolution,of,perturbations,about,this,mean,by,a,Gaussian,model,˜M,called,the,perturbation,forecast,(PF).,In,practice,,this,proceeds,by,introducing,and,iteratively,improving,a,‘guess’,xg,for,the,mean,of,the,new,state,(this,is,termed,the,incremental,approach).,Also,,in,the,absence,of,a,true,model,¯M,for,the,mean,of,the,PDF,,the,normal,NWP,model,M,is,used.,Now,the,evaluation,of,the,statistics,from,the,perturbation,forecast,model,form,the,inner,loop,and,this,needs,to,be,evaluable,without,recomputing,the,full,nonlinear,model,¯M,i.e.,it,uses,˜M,.,This,can,be,viewed,as,a,sort,of,reduced-order,model.,Another,critical,issue,in,NWP,is,the,so-called,butterﬂy,effect,in,which,errors,or,ﬂuctuations,at,small,scales,quickly,grow,and,affect,larger,scales.,This,means,that,current,NWP,algorithms,cannot,simply,be,expected,to,continue,to,work,at,a,higher,resolution,on,a,more,powerful,computer,without,some,degree,of,modiﬁcation.,A,solution,mentioned,in,[9],is,to,ﬁlter,the,PF,model,to,prevent,the,15,rapid,growth,of,the,affected,scales.,Another,issue,is,the,potentially,chaotic,nature,of,DA,systems,(and,note,that,deterministic,chaos,concerns,extreme,sensitivity,to,initial,conditions,,rather,than,the,coupling,between,scales,that,characterises,the,butterﬂy,effect).,This,chaotic,dynamics,becomes,worse,for,larger,systems,where,there,are,basically,not,enough,observations,to,control,the,growth,of,dynamics,governed,by,the,increasing,number,of,positive,Lyapunov,exponents.,More,detailed,expositions,of,4D-Var,can,be,found,in,[9],,[10].,16,4,Summary,This,report,has,provided,a,brief,overview,of,each,of,three,important,approaches,to,model,or-,der,reduction:,reduced,basis,methods,,proper,orthogonal,decomposition,and,proper,generalized,decomposition.,Speciﬁc,recommendations,for,methods,expected,to,work,well,in,the,magnetically-,conﬁned,fusion,(MCF),use,case,are,not,yet,possible,,though,it,is,clear,that,methods,capable,of,handling,nonlinearity,are,indicated,,inviting,a,further,study,of,POD,methods,and,also,indicating,the,need,for,techniques,such,as,empirical,interpolation,to,preserve,the,online,efﬁciency,of,reduced,basis,methods.,Another,,though,perhaps,longer-term,,consideration,is,the,requirement,for,an,MCF,control,system,to,be,able,to,quantify,and,mitigate,extreme,events,in,order,to,prevent,damage,to,a,fusion,machine,and,in,this,sense,,algorithms,of,the,greedy,type,are,useful,in,that,they,seek,to,identify,worst-case,events,rather,than,focussing,on,a,global,least-squares,minimization.,Existing,MOR,implementations,encapsulate,much,of,the,theory,and,computational,machinery,outlined,in,this,report,and,are,able,to,interface,with,external,PDE,solvers.,One,simple,initial,proposal,is,therefore,to,apply,tools,such,as,pyMOR,to,simulations,of,ﬂuid,turbulence,(a,simple,model,with,a,handful,of,inputs,and,a,single,main,physically-relevant,output,in,the,time-averaged,quasi-steady-state,heat,ﬂux,across,the,domain,but,a,large,number,of,internal,degrees,of,freedom);,such,models,are,a,proxy,for,heat,transport,near,the,outer,boundary,of,a,tokamak.,A,subsequent,section,presented,an,overview,of,data,assimilation,and,sketched,the,two,main,approaches,of,Kalman,ﬁltering,and,variational,data,assimilation.,Some,particular,issues,in,NWP,were,highlighted,,given,that,similar,problems,are,expected,in,the,case,of,tokamak,edge,physics,modelling,,key,shared,features,being,nonlinearity,,multiple,scales,and,turbulence;,the,main,interest,is,how,to,extend,the,techniques,of,DA,to,work,in,cases,where,the,errors,are,non-Gaussian,and,where,the,dynamical,model,is,not,linear.,It,also,revealed,that,NWP,models,use,what,is,spiritually,a,reduced-order,model,in,the,inner,loop,of,a,perturbation,forecast,model,for,estimating,model,variance,in,non-Gaussian,scenarios.,The,ensemble,Kalman,ﬁlter,was,highlighted,as,a,technique,for,ﬁtting,not,only,the,model,state,but,also,as,a,tool,for,Bayesian,parameter,estimation.,Acknowledgement,The,support,of,the,UK,Meteorological,Ofﬁce,and,Strategic,Priorities,Fund,is,acknowledged.,A,Gaussianity,The,Gaussian-distributed,random,variable,is,,in,a,sense,,the,simplest,random,quantity,,in,that,the,statistical,properties,are,speciﬁed,completely,the,the,ﬁrst,two,cumulants,,these,representing,the,mean,and,the,variance,of,the,probability,distribution.,Higher,cumulants,are,zero;,the,Gaussian,is,the,only,distribution,with,this,property,(indeed,,even,a,uniform,probability,distribution,has,an,inﬁnite,set,of,non-zero,cumulants).,The,well-known,Central,Limit,Theorem,states,that,a,linear,combination,of,a,large,number,of,un-,17,correlated,random,variables,tends,to,a,Gaussian-distributed,random,variable;,indeed,,any,linear,function,of,a,Gaussian,random,variable,is,itself,a,Gaussian,random,variable,i.e.,there,is,no,need,to,take,the,limit,in,this,case,(this,means,that,linear,dynamical,models,preserve,the,Gaussianity,of,errors,in,the,model,state).,This,can,be,expressed,in,the,fact,that,the,Gaussian,is,stable,under,lin-,ear,combination,of,like-distributed,variables.,There,are,other,stable,distributions,(for,example,,the,Cauchy,distribution),,but,the,Gaussian,is,the,only,one,with,ﬁnite,variance.,Note,that,it,has,been,demonstrated,in,the,main,text,that,the,Gaussian,is,also,stable,under,the,conditional,probability,rules,for,the,incorporation,of,new,observations,that,are,themselves,Gaussian,distributed.,If,an,N,-component,random,vector,x,is,Gaussian,,then,it,has,probability,density,p(x),=,1,(2π),N,2,|P,|−,1,2,exp,(cid:18),−,1,2,(x,−,¯x)T,P,−1(x,−,¯x),(cid:19),.,(36),Here,¯x,is,the,expectation,E(x),and,P,=,cov(x),≡,E((x,−,¯x)(x,−,¯x)T,).,Note,that,the,mode,of,the,Gaussian,distribution,is,equal,to,the,mean,(this,fact,means,that,the,mean,can,be,evaluated,by,minimizing,the,quadratic,argument,of,the,decaying,exponential,,or,equivalently,that,a,maximum-likelihood,estimate,gives,the,mean).,The,Fourier,transform,of,a,zero-mean,Gaussian,is,easily,evaluated,by,completing,the,square,,giving,F,T,(cid:18),1,√,2πσ2,e−x2/2σ2(cid:19),∝,e−σ2k2/2,,(37),showing,that,the,Fourier,transform,of,a,Gaussian,is,a,k-space,Gaussian.,References,[1],Matteo,Bonotto,,Paolo,Bettini,,and,Angelo,Cenedese.,Model,order,reduction,of,large-scale,state-space,models,in,fusion,machines,via,Krylov,methods.,IEEE,Transactions,on,Magnetics,,53,,2017.,[2],Julio,L.,Nicolini,,Dong-Yeop,Na,,and,Fernando,L.,Teixeira.,Model,order,reduction,of,elec-,tomagnetic,particle,-in-cell,kinetic,plasma,simulations,via,proper,orthogonal,decomposition.,IEEE,Transactions,on,Plasma,Science,,47,,2019.,[3],C.,Gillot,,G.,Dif-Pradalier,,X.,Garbet,,P.,Ghendrih,,V.,Grandgirard,,and,Y.,Sarazin.,Model,order,reduction,approach,to,the,one-dimensional,collisionless,closure,problem.,Physics,of,Plasmas,,28,,2021.,[4],Model,order,reduction,wiki.,https://en.wikipedia.org/wiki/Model_order_reduction,,2020.,Accessed:,October,2020.,[5],Francisco,Chinesta,,Antonio,Huerta,,Gianluigi,Rozza,,and,Karen,Willcox.,Model,Order,Re-,duction,,chapter,in,Encyclopedia,of,Computational,Mechanics.,John,Wiley,&,Sons,,2017.,18,[6],pyMOR,online,documentation.,https://docs.pymor.org,,2021.,Accessed:,March,2021.,[7],N.,Gershenfeld.,The,Nature,of,Mathematical,Modeling.,Cambridge,University,Press,,1999.,[8],C.,Snyder.,Introduction,to,the,Kalman,ﬁlter.,In,´E.,Blayo,,M.,Bocquet,,E.,Cosme,,and,L.,F.,Cugliandolo,,editors,,Advanced,Data,Assimilation,(Les,Houches,2012,Special,Issue),,chap-,ter,3,,page,75120.,Oxford,University,Press,,Oxford,,2012.,[9],A.C.,Lorenc.,Four-dimensional,variational,data,assimilation.,In,´E.,Blayo,,M.,Bocquet,,E.,Cosme,,and,L.,F.,Cugliandolo,,editors,,Advanced,Data,Assimilation,(Les,Houches,2012,Special,Issue),,chapter,2,,pages,31–73.,Oxford,University,Press,,Oxford,,2012.,[10],O.,Talagrand.,4D-VAR:,four-dimensional,variational,assimilation.,In,´E.,Blayo,,M.,Bocquet,,E.,Cosme,,and,L.,F.,Cugliandolo,,editors,,Advanced,Data,Assimilation,(Les,Houches,2012,Special,Issue),,chapter,1,,pages,3–30.,Oxford,University,Press,,Oxford,,2012.,19

:pdfembed:`src:_static/CD-EXCALIBUR-FMS0031-M2.5.1_SelectMorModelOrderReduction.pdf, height:1600, width:1100, align:middle`