ArangoDB v3.2.1 Documentation

TableofContents
Introduction
1.1
GettingStarted
1.2
Installing
1.2.1
Linux
1.2.1.1
M acOSX
1.2.1.2
Windows
1.2.1.3
Compiling
1.2.1.4
Authentication
1.2.2
AccessingtheWebInterface
1.2.3
ComingfromSQL
1.2.4
Highlights
1.3
Scalability
1.4
Architecture
1.4.1
Datamodels
1.4.2
Limitations
1.4.3
Datamodels&modeling
1.5
Concepts
1.5.1
Databases
1.5.2
WorkingwithDatabases
1.5.2.1
NotesaboutDatabases
1.5.2.2
Collections
1.5.3
CollectionM ethods
1.5.3.1
DatabaseM ethods
1.5.3.2
Documents
1.5.4
BasicsandTerminology
1.5.4.1
CollectionM ethods
1.5.4.2
DatabaseM ethods
1.5.4.3
Graphs,Vertices&Edges
1.5.5
NamingConventions
1.5.6
DatabaseNames
1.5.6.1
CollectionNames
1.5.6.2
DocumentKeys
1.5.6.3
AttributeNames
1.5.6.4
Indexing
1.6
IndexBasics
1.6.1
Whichindextousewhen
1.6.2
IndexUtilization
1.6.3
WorkingwithIndexes
1.6.4
HashIndexes
1.6.4.1
Skiplists
1.6.4.2
Persistent
1.6.4.3
1
FulltextIndexes
1.6.4.4
GeoIndexes
1.6.4.5
VertexCentricIndexes
1.6.4.6
Graphs
1.7
GeneralGraphs
1.7.1
GraphM anagement
1.7.1.1
GraphFunctions
1.7.1.2
SmartGraphs
SmartGraphM anagement
Traversals
1.7.2
1.7.2.1
1.7.3
UsingTraversalObjects
1.7.3.1
ExampleData
1.7.3.2
WorkingwithEdges
1.7.4
Pregel
1.7.5
FoxxM icroservices
1.8
Ataglance
1.8.1
Gettingstarted
1.8.2
Servicemanifest
1.8.3
Servicecontext
1.8.4
Configuration
1.8.5
Dependencies
1.8.6
Routers
1.8.7
Endpoints
1.8.7.1
M iddleware
1.8.7.2
Request
1.8.7.3
Response
1.8.7.4
UsingGraphQL
1.8.8
Sessionsmiddleware
1.8.9
Sessionstorages
1.8.9.1
Collectionstorage
1.8.9.1.1
JWTstorage
1.8.9.1.2
Sessiontransports
1.8.9.2
Cookietransport
1.8.9.2.1
Headertransport
1.8.9.2.2
Servingfiles
1.8.10
Writingtests
1.8.11
CrossOrigin
1.8.12
Scriptsandqueuedjobs
1.8.13
M igrating2.xservices
1.8.14
M igratingfrompre-2.8
1.8.14.1
manifest.json
1.8.14.2
applicationContext
1.8.14.3
RepositoriesandM odels
1.8.14.4
Controllers
1.8.14.5
2
Requestcontext
1.8.14.5.1
Errorhandling
1.8.14.5.2
Before/After/Around
1.8.14.5.3
Requestobject
1.8.14.5.4
Responseobject
1.8.14.5.5
DependencyInjection
1.8.14.5.6
Sessions
1.8.14.6
AuthandOAuth2
1.8.14.7
FoxxQueries
1.8.14.8
Legacycompatibilitymode
1.8.15
Usermanagement
1.8.16
Relatedmodules
1.8.17
Authentication
1.8.17.1
OAuth1.0a
1.8.17.2
OAuth2.0
1.8.17.3
Transactions
1.9
Transactioninvocation
1.9.1
Passingparameters
1.9.2
Lockingandisolation
1.9.3
Durability
1.9.4
Limitations
1.9.5
Deployment
1.10
Singleinstance
1.10.1
Cluster:M esos,DC/OS
1.10.2
Cluster:Generic&Docker
1.10.3
AdvancedTopics
1.10.4
StandaloneAgency
1.10.4.1
Cluster:Localtestsetups
1.10.4.2
Cluster:Processes
1.10.4.3
Cluster:Docker
1.10.4.4
Administration
WebInterface
1.11
1.11.1
Dashboard
1.11.1.1
Cluster
1.11.1.2
Collections
1.11.1.3
Document
1.11.1.4
Queries
1.11.1.5
Graphs
1.11.1.6
Services
1.11.1.7
Users
1.11.1.8
Logs
1.11.1.9
ArangoDBShell
1.11.2
ShellOutput
1.11.2.1
Configuration
1.11.2.2
3
Details
1.11.2.3
Arangoimp
1.11.3
Arangodump
1.11.4
Arangorestore
1.11.5
Arangoexport
1.11.6
M anagingUsers
1.11.7
InArangosh
ServerConfiguration
1.11.7.1
1.11.8
OperatingSystemConfiguration
1.11.8.1
M anagingEndpoints
1.11.8.2
SSLConfiguration
1.11.8.3
LDAPOptions
1.11.8.4
LoggingOptions
1.11.8.5
GeneralOptions
1.11.8.6
Write-AheadLogOptions
1.11.8.7
ClusterOptions
1.11.8.8
RocksDBEngineOptions
1.11.8.9
HashCacheOptions
1.11.8.10
AsynchronousTasks
1.11.8.11
Durability
1.11.9
Encryption
1.11.10
Auditing
1.11.11
Configuration
1.11.11.1
Events
1.11.11.2
Replication
1.11.12
AsynchronousReplication
1.11.12.1
Components
1.11.12.1.1
ExampleSetup
1.11.12.1.2
SyncingCollections
1.11.12.1.3
ReplicationLimitations
1.11.12.1.4
SynchronousReplication
1.11.12.2
Implementation
1.11.12.2.1
Configuration
1.11.12.2.2
SatelliteCollections
1.11.12.3
Sharding
1.11.13
Upgrading
1.11.14
Upgradingto3.2
1.11.14.1
Upgradingto3.1
1.11.14.2
Upgradingto3.0
1.11.14.3
Upgradingto2.8
1.11.14.4
Upgradingto2.6
1.11.14.5
Upgradingto2.5
1.11.14.6
Upgradingto2.4
1.11.14.7
Upgradingto2.3
1.11.14.8
4
Upgradingto2.2
Troubleshooting
1.11.14.9
1.12
arangod
1.12.1
EmergencyConsole
1.12.2
DatafileDebugger
1.12.3
Arangobench
1.12.4
Architecture
1.13
Write-aheadlog
1.13.1
StorageEngines
1.13.2
Releasenotes
1.14
WhatsNewin3.2
1.14.1
KnownIssuesin3.2
1.14.2
Incompatiblechangesin3.2
1.14.3
WhatsNewin3.1
1.14.4
Incompatiblechangesin3.1
1.14.5
WhatsNewin3.0
1.14.6
Incompatiblechangesin3.0
1.14.7
WhatsNewin2.8
1.14.8
Incompatiblechangesin2.8
1.14.9
WhatsNewin2.7
1.14.10
Incompatiblechangesin2.7
1.14.11
WhatsNewin2.6
1.14.12
Incompatiblechangesin2.6
1.14.13
WhatsNewin2.5
1.14.14
Incompatiblechangesin2.5
1.14.15
WhatsNewin2.4
1.14.16
Incompatiblechangesin2.4
1.14.17
WhatsNewin2.3
1.14.18
Incompatiblechangesin2.3
1.14.19
WhatsNewin2.2
1.14.20
WhatsNewin2.1
1.14.21
Appendix
References
1.15
1.15.1
db
1.15.1.1
collection
1.15.1.2
JavaScriptM odules
1.15.2
@arangodb
1.15.2.1
console
1.15.2.2
crypto
1.15.2.3
fs
1.15.2.4
request
1.15.2.5
actions
1.15.2.6
queries
1.15.2.7
Write-aheadlog
1.15.2.8
5
TaskM anagement
Deprecated
SimpleQueries
1.15.2.9
1.15.3
1.15.3.1
SequentialAccess
1.15.3.1.1
Pagination
1.15.3.1.2
M odificationQueries
1.15.3.1.3
GeoQueries
1.15.3.1.4
FulltextQueries
1.15.3.1.5
Actions
1.15.3.2
DeliveringHTM LPages
1.15.3.2.1
JsonObjects
1.15.3.2.2
M odifying
1.15.3.2.3
Errorcodesandmeanings
1.15.4
Glossary
1.15.5
6
Introduction
ArangoDBv3.2.1Documentation
WelcometotheArangoDBdocumentation!
NewandeagertotryoutArangoDB?Startrightawaywithourbeginner'sguide:GettingS tarted
Thedocumentationisorganizedinfourhandbooks:
ThismanualdescribesArangoDBanditsfeaturesindetailforyouasauser,developerandadministrator.
TheAQLhandbookexplainsArangoDB'squerylanguageAQL.
TheHTTPhandbookdescribestheinternalAPIofArangoDBthatisusedtocommunicatewithclients.Ingeneral,theHTTP
handbookwillbeofinteresttodriverdevelopers.Ifyouuseanyoftheexistingdriversforthelanguageofyourchoice,youcanskip
thishandbook.
Ourcookbookwithrecipesforspecificproblemsandsolutions.
Featuresareillustratedwithinteractiveusageexamples;youcancut'n'pastethemintoarangoshtotrythemout.TheHTTPREST-API
fordriverdevelopersisdemonstratedwithcut'n'pasterecepiesintendedtobeusedwiththecURL.Driversmayprovidetheirown
examplesbasedonthese.jsbasedexamplestoimproveunderstandeabilityfortheirrespectiveusers,i.e.forthejavadriversomeofthe
samplesarere-implemented.
Overview
ArangoDBisanativemulti-model,open-sourcedatabasewithflexibledatamodelsfordocuments,graphs,andkey-values.Buildhigh
performanceapplicationsusingaconvenientSQL-likequerylanguageorJavaScriptextensions.UseACIDtransactionsifyourequire
them.Scalehorizontallyandverticallywithafewmouseclicks.
Keyfeaturesinclude:
installingArangoDBonaclusterisaseasyasinstallinganapponyourmobile
Flexibledatamodeling:modelyourdataascombinationofkey-valuepairs,documentsorgraphs-perfectforsocialrelations
Powerfulquerylanguage(AQL)toretrieveandmodifydata
UseArangoDBasanapplicationserverandfuseyourapplicationanddatabasetogetherformaximalthroughput
Transactions:runqueriesonmultipledocumentsorcollectionswithoptionaltransactionalconsistencyandisolation
ReplicationandS harding:setupthedatabaseinamaster-slaveconfigurationorspreadbiggerdatasetsacrossmultipleservers
Configurabledurability:lettheapplicationdecideifitneedsmoredurabilityormoreperformance
No-nonsensestorage:ArangoDBusesallofthepowerofmodernstoragehardware,likeSSDandlargecaches
JavaScriptforall:nolanguagezoo,youcanuseonelanguagefromyourbrowsertoyourback-end
ArangoDBcanbeeasilydeployedasafault-tolerantdistributedstatemachine,whichcanserveastheanimalbrainof
distributedappliances
Itisopensource(ApacheLicense2.0)
Community
IfyouhavequestionsregardingArangoDB,Foxx,drivers,orthisdocumentationdon'thesitatetocontactuson:
GitHubforissuesandmisbehaviororpullrequests
GoogleGroupsfordiscussionsaboutArangoDBingeneralortoannounceyournewFoxxApp
StackOverflowforquestionsaboutAQL,usagescenariosetc.
Slack,ourcommunitychat
Whenreportingissues,pleasedescribe:
theenvironmentyourunArangoDBin
theArangoDBversionyouuse
whetheryou'reusingFoxx
theclientyou'reusing
7
Introduction
whichpartsofthedocumentationyou'reworkingwith(link)
whatyouexpecttohappen
whatisactuallyhappening
Wewillrespondassoonaspossible.
8
GettingStarted
Gettingstarted
Overview
Thisbeginner'sguidewillmakeyoufamiliarwithArangoDB.Wewillcoverhowto
installandrunalocalArangoDBserver
usethewebinterfacetointeractwithit
storeexampledatainthedatabase
querythedatabasetoretrievethedataagain
editandremoveexistingdata
Installation
Headtoarangodb.com/download,selectyouroperatingsystemanddownloadArangoDB.Youmayalsofollowtheinstructionsonhow
toinstallwithapackagemanager,ifavailable.
IfyouinstalledabinarypackageunderLinux,theserverisautomaticallystarted.
IfyouinstalledArangoDBusinghomebrewunderM acOSX,starttheserverbyrunning /usr/local/sbin/arangod.
IfyouinstalledArangoDBunderWindowsasaservice,theserverisautomaticallystarted.Otherwise,runthe arangod.exelocatedin
theinstallationfolder's bindirectory.Youmayhavetorunitasadministratortograntitwritepermissionsto C:\ProgramFiles.
Formorein-depthinformationonhowtoinstallArangoDB,aswellasavailablestartupparameters,installationinaclusterandsoon,see
Installing.
Wanthelp?Readmoreaboutconsulting,traininganddevelopersupport
Securingtheinstallation
Thedefaultinstallationcontainsonedatabase_systemandausernamedroot.
DebianbasedpackagesandtheWindowsinstallerwillaskforapasswordduringtheinstallationprocess.Red-Hatbasedpackageswill
setarandompassword.Forallotherinstallationpackagesyouneedtoexecute
shell>arango-secure-installation
Thiswillaskedforarootpasswordandsetsthispassword.
Webinterface
Theserveritself(arangod)speaksHTTP/REST,butyoucanusethegraphicalwebinterfacetokeepitsimple.There'salsoarangosh,a
synchronousshellforinteractionwiththeserver.Ifyou'readeveloper,youmightprefertheshellovertheGUI.Itdoesnotprovide
featureslikesyntaxhighlightinghowever.
WhenyoustartusingArangoDBinyourproject,youwilllikelyuseanofficialorcommunity-madedriverwritteninthesamelanguageas
yourproject.Driversimplementaprogramminginterfacethatshouldfeelnaturalforthatprogramminglanguage,anddoallthetalkingto
theserver.Therefore,youcanmostcertainlyignoretheHTTPAPIunlessyouwanttowriteadriveryourselforexplicitlywanttouse
therawinterface.
Togetfamiliarwiththedatabasesystemyoucanevenputdriversasideandusethewebinterface(codenameAardvark)forbasic
interaction.Thewebinterfacewillbecomeavailableshortlyafteryoustarted arangod.Youcanaccessitinyourbrowserat
http://localhost:8529-ifnot,pleaseseeTroubleshooting.
Bydefault,authenticationisenabled.Thedefaultuseris root.Dependingontheinstallationmethodused,theinstallationprocess
eitherpromptedfortherootpasswordorthedefaultrootpasswordisempty(seeabove).
9
GettingStarted
Nextyouwillbeaskedwhichdatabasetouse.Everyserverinstancecomeswitha _systemdatabase.Selectthisdatabasetocontinue.
Youshouldthenbepresentedthedashboardwithserverstatisticslikethis:
10
GettingStarted
Foramoredetaileddescriptionoftheinterface,seeWebInterface.
Databases,collectionsanddocuments
Databasesaresetsofcollections.Collectionsstorerecords,whicharereferredtoasdocuments.Collectionsaretheequivalentoftablesin
RDBM S,anddocumentscanbethoughtofasrowsinatable.Thedifferenceisthatyoudon'tdefinewhatcolumns(orratherattributes)
therewillbeinadvance.Everydocumentinanycollectioncanhavearbitraryattributekeysandvalues.Documentsinasinglecollection
willlikelyhaveasimilarstructureinpracticehowever,butthedatabasesystemitselfdoesnotimposeitandwilloperatestableandfast
nomatterhowyourdatalookslike.
Readmoreinthedata-modelconceptschapter.
Fornow,youcanstickwiththedefault _systemdatabaseandusethewebinterfacetocreatecollectionsanddocuments.Startby
clickingtheCOLLECTIONSmenuentry,thentheAddCollectiontile.Giveitaname,e.g.users,leavetheothersettingsunchanged(we
wantittobeadocumentcollection)andSaveit.Anewtilelabeledusersshouldshowup,whichyoucanclicktoopen.
TherewillbeNodocumentsyet.Clickthegreencirclewiththewhiteplusontheright-handsidetocreateafirstdocumentinthis
collection.Adialogwillaskyoufora _key.YoucanleavethefieldblankandclickCreatetoletthedatabasesystemassignan
automaticallygenerated(unique)key.Notethatthe _keypropertyisimmutable,whichmeansyoucannotchangeitoncethedocument
iscreated.Whatyoucanuseasdocumentkeyisdescribedinthenamingconventions.
Anautomaticallygeneratedkeycouldbe "9883"( _keyisalwaysastring!),andthedocument _idwouldbe "users/9883"inthat
case.Asidefromafewsystemattributes,thereisnothinginthisdocumentyet.Let'saddacustomattributebyclickingtheicontothe
leftof(emptyobject),thenAppend.Twoinputfieldswillbecomeavailable,FIELD(attributekey)andVALUE(attributevalue).Type
nameaskeyandyournameasvalue.Appendanotherattribute,nameit ageandsetittoyourage.ClickSavetopersistthechanges.If
youclickonCollection:usersatthetopontheright-handsideoftheArangoDBlogo,thedocumentbrowserwillshowthedocumentsin
theuserscollectionandyouwillseethedocumentyoujustcreatedinthelist.
Queryingthedatabase
11
GettingStarted
TimetoretrieveourdocumentusingAQL,ArangoDB'squerylanguage.Wecandirectlylookupthedocumentwecreatedviathe _id,
buttherearealsootheroptions.ClicktheQUERIESmenuentrytobringupthequeryeditorandtypethefollowing(adjustthe
documentIDtomatchyourdocument):
RETURNDOCUMENT("users/9883")
ThenclickExecutetorunthequery.Theresultappearsbelowthequeryeditor:
[
{
"_key":"9883",
"_id":"users/9883",
"_rev":"9883",
"age":32,
"name":"JohnSmith"
}
]
Asyoucansee,theentiredocumentincludingthesystemattributesisreturned.DOCUM ENT()isafunctiontoretrieveasingle
documentoralistofdocumentsofwhichyouknowthe _keysor _ids.Wereturntheresultofthefunctioncallasourqueryresult,
whichisourdocumentinsideoftheresultarray(wecouldhavereturnedmorethanoneresultwithadifferentquery,butevenforasingle
documentasresult,westillgetanarrayatthetoplevel).
Thistypeofqueryiscalleddataaccessquery.Nodataiscreated,changedordeleted.Thereisanothertypeofquerycalleddata
modificationquery.Let'sinsertaseconddocumentusingamodificationquery:
INSERT{name:"KatieFoster",age:27}INTOusers
Thequeryisprettyself-explanatory:the INSERTkeywordtellsArangoDBthatwewanttoinsertsomething.Whattoinsert,a
documentwithtwoattributesinthiscase,followsnext.Thecurlybraces {}signifydocuments,orobjects.Whentalkingabout
recordsinacollection,wecallthemdocuments.EncodedasJSON,wecallthemobjects.Objectscanalsobenested.Here'sanexample:
{
"name":{
"first":"Katie",
"last":"Foster"
}
}
INTOisamandatorypartofevery INSERToperationandisfollowedbythecollectionnamethatwewanttostorethedocumentin.
Notethattherearenoquotemarksaroundthecollectionname.
Ifyourunabovequery,therewillbeanemptyarrayasresultbecausewedidnotspecifywhattoreturnusinga RETURNkeyword.Itis
optionalinmodificationqueries,butmandatoryindataaccessqueries.Evenwith RETURN,thereturnvaluecanstillbeanemptyarray,
e.g.ifthespecifieddocumentwasnotfound.Despitetheemptyresult,theabovequerystillcreatedanewuserdocument.Youcanverify
thiswiththedocumentbrowser.
Let'saddanotheruser,butreturnthenewlycreateddocumentthistime:
INSERT{name:"JamesHendrix",age:69}INTOusers
RETURNNEW
NEWisapseudo-variable,whichreferstothedocumentcreatedby INSERT.Theresultofthequerywilllooklikethis:
[
{
"_key":"10074",
"_id":"users/10074",
"_rev":"10074",
"age":69,
"name":"JamesHendrix"
}
]
12
GettingStarted
Nowthatwehave3usersinourcollection,howtoretrievethemallwithasinglequery?Thefollowingdoesnotwork:
RETURNDOCUMENT("users/9883")
RETURNDOCUMENT("users/9915")
RETURNDOCUMENT("users/10074")
Therecanonlybeasingle RETURNstatementhereandasyntaxerrorisraisedifyoutrytoexecuteit.The DOCUMENT()functionoffersa
secondarysignaturetospecifymultipledocumenthandles,sowecoulddo:
RETURNDOCUMENT(["users/9883","users/9915","users/10074"])
Anarraywiththe _idsofall3documentsispassedtothefunction.Arraysaredenotedbysquarebrackets []andtheirelements
areseparatedbycommas.
Butwhatifweaddmoreusers?Wewouldhavetochangethequerytoretrievethenewlyaddedusersaswell.Allwewanttosaywith
ourqueryis:"Foreveryuserinthecollectionusers,returntheuserdocument".Wecanformulatethiswitha FORloop:
FORuserINusers
RETURNuser
Itexpressestoiterateovereverydocumentin usersandtouse userasvariablename,whichwecanusetorefertothecurrentuser
document.Itcouldalsobecalled doc, uor ahuacatlguacamole,thisisuptoyou.Itisadvisabletouseashortandself-descriptive
namehowever.
Theloopbodytellsthesystemtoreturnthevalueofthevariable user,whichisasingleuserdocument.Alluserdocumentsare
returnedthisway:
[
{
"_key":"9915",
"_id":"users/9915",
"_rev":"9915",
"age":27,
"name":"KatieFoster"
},
{
"_key":"9883",
"_id":"users/9883",
"_rev":"9883",
"age":32,
"name":"JohnSmith"
},
{
"_key":"10074",
"_id":"users/10074",
"_rev":"10074",
"age":69,
"name":"JamesHendrix"
}
]
Youmayhavenoticedthattheorderofthereturneddocumentsisnotnecessarilythesameastheywereinserted.Thereisnoorder
guaranteedunlessyouexplicitlysortthem.Wecanadda SORToperationveryeasily:
FORuserINusers
SORTuser._key
RETURNuser
Thisdoesstillnotreturnthedesiredresult:James(10074)isreturnedbeforeJohn(9883)andKatie(9915).Thereasonisthatthe _key
attributeisastringinArangoDB,andnotanumber.Theindividualcharactersofthestringsarecompared. 1islowerthan 9andthe
resultistherefore"correct".Ifwewantedtousethenumericalvalueofthe _keyattributesinstead,wecouldconvertthestringtoa
numberanduseitforsorting.Therearesomeimplicationshowever.Wearebetteroffsortingsomethingelse.Howabouttheage,in
descendingorder?
13
GettingStarted
FORuserINusers
SORTuser.ageDESC
RETURNuser
Theuserswillbereturnedinthefollowingorder:James(69),John(32),Katie(27).Insteadof DESCfordescendingorder, ASCcanbe
usedforascendingorder. ASCisthedefaultthoughandcanbeomitted.
Wemightwanttolimittheresultsettoasubsetofusers,basedontheageattributeforexample.Let'sreturnusersolderthan30only:
FORuserINusers
FILTERuser.age>30
SORTuser.age
RETURNuser
ThiswillreturnJohnandJames(inthisorder).Katie'sageattributedoesnotfulfillthecriterion(greaterthan30),sheisonly27and
thereforenotpartoftheresultset.Wecanmakeheragetoreturnheruserdocumentagain,usingamodificationquery:
UPDATE"9915"WITH{age:40}INusers
RETURNNEW
UPDATEallowstopartiallyeditanexistingdocument.Thereisalso REPLACE,whichwouldremoveallattributes(exceptfor _keyand
_id,whichremainthesame)andonlyaddthespecifiedones. UPDATEontheotherhandonlyreplacesthespecifiedattributesand
keepseverythingelseas-is.
The UPDATEkeywordisfollowedbythedocumentkey(oradocument/objectwitha _keyattribute)toidentifywhattomodify.The
attributestoupdatearewrittenasobjectafterthe WITHkeyword. INdenotesinwhichcollectiontoperformthisoperationin,just
like INTO(bothkeywordsareactuallyinterchangablehere).Thefulldocumentwiththechangesappliedisreturnedifweusethe NEW
pseudo-variable:
[
{
"_key":"9915",
"_id":"users/9915",
"_rev":"12864",
"age":40,
"name":"KatieFoster"
}
Ifweused REPLACEinstead,thenameattributewouldbegone.With UPDATE,theattributeiskept(thesamewouldapplytoadditional
attributesifwehadthem).
Letusrunour FILTERqueryagain,butonlyreturntheusernamesthistime:
FORuserINusers
FILTERuser.age>30
SORTuser.age
RETURNuser.name
Thiswillreturnthenamesofall3users:
[
"JohnSmith",
"KatieFoster",
"JamesHendrix"
]
Itiscalledaprojectionifonlyasubsetofattributesisreturned.Anotherkindofprojectionistochangethestructureoftheresults:
FORuserINusers
RETURN{userName:user.name,age:user.age}
14
GettingStarted
Thequerydefinestheoutputformatforeveryuserdocument.Theusernameisreturnedas userNameinsteadof name,theagekeeps
theattributekeyinthisexample:
[
{
"userName":"JamesHendrix",
"age":69
},
{
"userName":"JohnSmith",
"age":32
},
{
"userName":"KatieFoster",
"age":40
}
]
Itisalsopossibletocomputenewvalues:
FORuserINusers
RETURNCONCAT(user.name,"'sageis",user.age)
CONCAT()isafunctionthatcanjoinelementstogethertoastring.Weuseitheretoreturnastatementforeveryuser.Asyoucansee,the
resultsetdoesnotalwayshavetobeanarrayofobjects:
[
"JamesHendrix'sageis69",
"JohnSmith'sageis32",
"KatieFoster'sageis40"
]
Nowlet'sdosomethingcrazy:foreverydocumentintheuserscollection,iterateoveralluserdocumentsagainandreturnuserpairs,e.g.
JohnandKatie.Wecanusealoopinsidealoopforthistogetthecrossproduct(everypossiblecombinationofalluserrecords,3*3=
9).Wedon'twantpairingslikeJohn+Johnhowever,solet'seliminatethemwithafiltercondition:
FORuser1INusers
FORuser2INusers
FILTERuser1!=user2
RETURN[user1.name,user2.name]
Weget6pairings.PairslikeJames+JohnandJohn+Jamesarebasicallyredundant,butfairenough:
[
["JamesHendrix","JohnSmith"],
["JamesHendrix","KatieFoster"],
["JohnSmith","JamesHendrix"],
["JohnSmith","KatieFoster"],
["KatieFoster","JamesHendrix"],
["KatieFoster","JohnSmith"]
]
Wecouldcalculatethesumofbothagesandcomputesomethingnewthisway:
FORuser1INusers
FORuser2INusers
FILTERuser1!=user2
RETURN{
pair:[user1.name,user2.name],
sumOfAges:user1.age+user2.age
}
Weintroduceanewattribute sumOfAgesandaddupbothagesforthevalue:
[
15
GettingStarted
{
"pair":["JamesHendrix","JohnSmith"],
"sumOfAges":101
},
{
"pair":["JamesHendrix","KatieFoster"],
"sumOfAges":109
},
{
"pair":["JohnSmith","JamesHendrix"],
"sumOfAges":101
},
{
"pair":["JohnSmith","KatieFoster"],
"sumOfAges":72
},
{
"pair":["KatieFoster","JamesHendrix"],
"sumOfAges":109
},
{
"pair":["KatieFoster","JohnSmith"],
"sumOfAges":72
}
]
Ifwewantedtopost-filteronthenewattributetoonlyreturnpairswithasumlessthan100,weshoulddefineavariabletotemporarily
storethesum,sothatwecanuseitina FILTERstatementaswellasinthe RETURNstatement:
FORuser1INusers
FORuser2INusers
FILTERuser1!=user2
LETsumOfAges=user1.age+user2.age
FILTERsumOfAges<100
RETURN{
pair:[user1.name,user2.name],
sumOfAges:sumOfAges
}
The LETkeywordisfollowedbythedesignatedvariablename( sumOfAges),thenthere'sa =symbolandthevalueoranexpression
todefinewhatvaluethevariableissupposedtohave.Were-useourexpressiontocalculatethesumhere.Wethenhaveanother FILTER
toskiptheunwantedpairingsandmakeuseofthevariablewedeclaredbefore.Wereturnaprojectionwithanarrayoftheusernamesand
thecalculatedage,forwhichweusethevariableagain:
[
{
"pair":["JohnSmith","KatieFoster"],
"sumOfAges":72
},
{
"pair":["KatieFoster","JohnSmith"],
"sumOfAges":72
}
]
Protip:whendefiningobjects,ifthedesiredattributekeyandthevariabletousefortheattributevaluearethesame,youcanusea
shorthandnotation: {sumOfAges}insteadof {sumOfAges:sumOfAges}.
Finally,let'sdeleteoneoftheuserdocuments:
REMOVE"9883"INusers
ItdeletestheuserJohn( _key:"9883").Wecouldalsoremovedocumentsinaloop(samegoesfor INSERT, UPDATEand REPLACE):
FORuserINusers
FILTERuser.age>=30
REMOVEuserINusers
16
GettingStarted
Thequerydeletesalluserswhoseageisgreaterthanorequalto30.
Howtocontinue
ThereisalotmoretodiscoverinAQLandmuchmorefunctionalitythatArangoDBoffers.Continuereadingtheotherchaptersand
experimentwithatestdatabasetofosteryourknowledge.
IfyouwanttowritemoreAQLqueriesrightnow,havealookhere:
DataQueries:dataaccessandmodificationqueries
High-leveloperations:detaileddescriptionsof FOR, FILTERandmoreoperationsnotshowninthisintroduction
Functions:areferenceofallprovidedfunctions
ArangoDBprograms
TheArangoDBpackagecomeswiththefollowingprograms:
arangod:TheArangoDBdatabasedaemon.Thisserverprogramisintendedtorunasadaemonprocessandtoservethevarious
clientsconnectiontotheserverviaTCP/HTTP.
arangosh:TheArangoDBshell.Aclientthatimplementsaread-eval-printloop(REPL)andprovidesfunctionstoaccessand
administratetheArangoDBserver.
arangoimp:AbulkimporterfortheArangoDBserver.ItsupportsJSONandCSV.
arangodump:AtooltocreatebackupsofanArangoDBdatabaseinJSONformat.
arangorestore:AtooltoloaddataofabackupbackintoanArangoDBdatabase.
arango-dfdb:AdatafiledebuggerforArangoDB.ItisprimarilyintendedtobeusedduringdevelopmentofArangoDB.
arangobench:Abenchmarkandtesttool.Itcanbeusedforperformanceandserverfunctiontesting.
17
Installing
Installing
Firstofall,downloadandinstallthecorrespondingRPM orDebianpackageorusehomebrewonM acOSX.Youcanfindpackagesfor
variousoperationsystemsatourinstallsection,includinginstallersforWindows.
Howtodothatindetailisdescribedinthesubchaptersofthissection.
Onhowtosetupacluster,checkouttheDeploymentchapter.
18
Linux
Linux
VisittheofficialArangoDBinstallpageanddownloadthecorrectpackageforyourLinuxdistribution.Youcanfindbinarypackages
forthemostcommondistributionsthere.
Followtheinstructionstouseyourfavoritepackagemanagerforthemajordistributions.AftersettinguptheArangoDBrepository
youcaneasilyinstallArangoDBusingyum,aptitude,urpmiorzypper.
Debianbasedpackageswillaskforapasswordduringinstallation.ForanunattendedinstallationforDebian,seebelow.Red-Hat
basedpackageswillsetarandompasswordduringinstallation.Forotherdistributionsortochangethepassword,run arangosecure-installationtosetarootpassword.
Alternatively,seeCompilingifyouwanttobuildArangoDByourself.
Startupthedatabaseserver.
Normally,thisisdonebyexecutingthefollowingcommand:
unix>/etc/init.d/arangodstart
Itwillstarttheserver,anddothataswellatsystemboottime.
Tostoptheserveryoucanusethefollowingcommand:
unix>/etc/init.d/arangodstop
TheexactcommandsdependonyourLinuxdistribution.Youmayrequirerootprivilegestoexecutethesecommands.
LinuxMint
PleaseusethecorrespondingUbuntuorDebianpackages.
UnattendedInstallation
Debianbasedpackagewillaskforapasswordduringinstallation.Forunattendedinstallation,youcansetthepasswordusingthedebconf
helpers.
echoarangodb3arangodb3/passwordpasswordNEWPASSWORD|debconf-set-selections
echoarangodb3arangodb3/password_againpasswordNEWPASSWORD|debconf-set-selections
Thecommandsshouldbeexecutedpriortotheinstallation.
Red-Hatbasedpackageswillsetarandompasswordduringinstallation.Ifyouwanttoforceapassword,execute
ARANGODB_DEFAULT_ROOT_PASSWORD=NEWPASSWORDarango-secure-installation
Thecommandshouldbeexecutedaftertheinstallation.
Non-StandardInstallation
IfyoucompiledArangoDBfromsourceanddidnotuseanyinstallationpackage–orusingnon-defaultlocationsand/ormultiple
ArangoDBinstancesonthesamehost–youmaywanttostarttheserverprocessmanually.Youcandosobyinvokingthearangod
binaryfromthecommandlineasshownbelow:
unix>/usr/local/sbin/arangod/tmp/vocbase
20ZZ-XX-YYT12:37:08Z[8145]INFOusingbuilt-inJavaScriptstartupfiles
20ZZ-XX-YYT12:37:08Z[8145]INFOArangoDB(version1.x.y)isreadyforbusiness
20ZZ-XX-YYT12:37:08Z[8145]INFOHaveFun!
19
Linux
Tostopthedatabaseservergracefully,youcaneitherpressCTRL-CorbysendtheSIGINTsignaltotheserverprocess.Onmany
systemsthiscanbeachievedwiththefollowingcommand:
unix>kill-2`pidofarangod`
Onceyoustartedtheserver,thereshouldbearunninginstanceofarangod-theArangoDBdatabaseserver.
unix>psauxw|fgreparangod
arangodb145360.10.6530726423464s002S1:21pm0:00.18/usr/local/sbin/arangod
Ifthereisnosuchprocess,checkthelogfile/var/log/arangodb/arangod.logforerrors.Ifyouseealogmessagelike
2012-12-03T11:35:29Z[12882]ERRORDatabasedirectoryversion(1)islowerthanserverversion(1.2).
2012-12-03T11:35:29Z[12882]ERRORItseemslikeyouhaveupgradedtheArangoDBbinary.Ifthisiswhatyouwantedtodo,pleas
erestartwiththe--database.auto-upgradeoptiontoupgradethedatainthedatabasedirectory.
2012-12-03T11:35:29Z[12882]FATALDatabaseversioncheckfailed.Pleasestarttheserverwiththe--database.auto-upgradeopti
on
makesuretostarttheserveroncewiththe--database.auto-upgradeoption.
Notethatyoumayhavetoenableloggingfirst.Ifyoustarttheserverinashell,youshouldseeerrorsloggedthereaswell.
20
M acOSX
MacOSX
ThepreferredmethodforinstallingArangoDBunderM acOSXishomebrew.However,incaseyouarenotusinghomebrew,weprovide
acommand-lineapporgraphicalappwhichcontainsalltheexecutables.
Homebrew
Ifyouareusinghomebrew,thenyoucaninstallthelatestreleasedstableversionofArangoDBusingbrewasfollows:
brewinstallarangodb
ThiswillinstallthecurrentstableversionofArangoDBandalldependencieswithinyourHomebrewtree.Notethattheserverwillbe
installedas:
/usr/local/sbin/arangod
Youcanstarttheserverbyrunningthecommand /usr/local/sbin/arangod&.
Configurationfileislocatedat
/usr/local/etc/arangodb3/arangod.conf
TheArangoDBshellwillbeinstalledas:
/usr/local/bin/arangosh
YoucanuninstallArangoDBusing:
brewuninstallarangodb
However,incaseyoustartedArangoDBusingthelaunchctl,youneedtounloaditbeforeuninstallingtheserver:
launchctlunload~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist
ThenremovetheLaunchAgent:
rm~/Library/LaunchAgents/homebrew.mxcl.arangodb.plist
Note:IfthelatestArangoDBVersionisnotshowninhomebrew,youalsoneedtoupdatehomebrew:
brewupdate
Knownissues
Performance-theLLVM deliveredasofM acOSXElCapitanbuildsslowbinaries.UseGCCinstead,untilthisissuehasbeenfixed
byApple.
theCommandlineargumentparsingdoesn'tacceptblanksinfilenames;theCLIversionbelowdoes.
ifyouneedtochangeserverendpointwhilestartinghomebrewversion,youcaneditarangod.conffileanduncommentlinewith
endpointneeded,e.g.:
[server]
endpoint=tcp://0.0.0.0:8529
21
M acOSX
GraphicalApp
Incaseyouarenotusinghomebrew,wealsoprovideagraphicalapp.Youcandownloaditfromhere.
ChooseMacOSX.DownloadandinstalltheapplicationArangoDBinyourapplicationfolder.
CommandlineApp
Incaseyouarenotusinghomebrew,wealsoprovideacommand-lineapp.Youcandownloaditfromhere.
ChooseMacOSX.DownloadandinstalltheapplicationArangoDB-CLIinyourapplicationfolder.
Startingtheapplicationwillstarttheserverandopenaterminalwindowshowingyouthelog-file.
ArangoDBserverhasbeenstarted
Thedatabasedirectoryislocatedat
'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/lib/arangodb'
Thelogfileislocatedat
'/Applications/ArangoDB-CLI.app/Contents/MacOS/opt/arangodb/var/log/arangodb/arangod.log'
Youcanaccesstheserverusingabrowserat'http://127.0.0.1:8529/'
orstarttheArangoDBshell
'/Applications/ArangoDB-CLI.app/Contents/MacOS/arangosh'
Switchingtolog-filenow,killingthiswindowswillNOTstoptheserver.
2013-10-27T19:42:04Z[23840]INFOArangoDB(version1.4.devel[darwin])isreadyforbusiness.Havefun!
Notethatitispossibletoinstallboth,thehomebrewversionandthecommand-lineapp.Youshould,however,edittheconfigurationfiles
ofoneversionandchangetheportused.
22
Windows
Windows
ThedefaultinstallationdirectoryisC:\ProgramFiles\ArangoDB-3.x.x.Duringtheinstallationprocessyoumaychangethis.Inthe
followingdescriptionwewillassumethatArangoDBhasbeeninstalledinthelocation<ROOTDIR>.
Youhavetobecarefulwhenchoosinganinstallationdirectory.Youneedeitherwritepermissiontothisdirectoryoryouneedtomodify
theconfigurationfilefortheserverprocess.InthelattercasethedatabasedirectoryandtheFoxxdirectoryhavetobewritablebythe
user.
Single-andMultiuserInstallation
TherearetwomainmodesfortheinstallerofArangoDB.Theinstallerletsyouselect:
multiuserinstallation(default;adminprivilegesrequired)WillinstallArangoDBasservice.
singleuserinstallationAllowtoinstallArangodbasnormaluser.Requiresmanualstartingofthedatabaseserver.
CheckBoxes
Thecheckboxesallowyoutochoseweatheryouwantto:
chosecustominstallpaths
doanautomaticupgrade
keepanbackupofyourdata
addexecutablestopath
createadesktopicon
ornot.
CustomInstallPaths
Thischeckboxcontrolsifyouwillbeabletooverridethedefaultpathsfortheinstallationinsubsequentsteps.
Thedefaultinstallationpathsare:
M ultiUserDefault:
Installation:C:\ProgramFiles\ArangoDB-3.x.x
DataBase:C:\ProgramData\ArangoDB
FoxxService:C:\ProgramData\ArangoDB-apps
SingleUserDefault:
Installation:C:\Users\\\AppData\Local\ArangoDB-3.x.x
DataBase:C:\Users\\\AppData\Local\ArangoDB
FoxxService:C:\Users\\\AppData\Local\ArangoDB-apps
Wearenotusingtheroamingpartoftheuser'sprofile,becausedoingsoavoidsthedatabeingsyncedtothewindowsdomaincontroller.
AutomaticUpgrade
Ifthischeckboxisselectedtheinstallerwillattempttoperformanautomaticupdate.FormoreinformationpleaseseeUpgradingfrom
PreviousVersion.
KeepBackup
Selectthistocreateabackupofyourdatabasedirectoryduringautomaticupgrade.Thebackupwillbecreatednexttoyourcurrent
databasedirectorysuffixedbyatimestamp.
23
Windows
AddtoPath
Selectthistoaddthebinarydirectorytoyoursystem'spath(multiuserinstallation)oruser'spath(singleuserinstallation).
DesktopIcon
SelectifyouwanttheinstallertocreateDesktopIconsthatletyou:
accessthewebinteface
startthecommandlineclient(arangosh)
startthedatabaseserver(singleuserinstallationonly)
UpgradingfromPreviousVersion
IfyouareupgradingArangoDBfromanearlierversionyouneedtocopyyourolddatabasedirectorytothenewdefaultpaths.
Upgradingwillkeepyourolddata,passwordandchoiceofstorageengineasitis.SwitchingtotheRocksDBstorageenginerequiresa
exportandreimportofyourdata.
Starting
IfyouinstalledArangoDBformultipleusers(asaservice)itisautomaticallystarted.Otherwiseyouneedtousethelinkthatwas
createdonyouDesktopifyouchosetolettheinstallercreatedesktopiconsor
theexecutablearangod.exelocatedin<ROOTDIR>\bin.Thiswillusetheconfigurationfilearangod.conflocatedin
<ROOTDIR>\etc\arangodb,whichyoucanadjusttoyourneedsandusethedatadirectory<ROOTDIR>\var\lib\arangodb.Thisisthe
placewhereallyourdata(databasesandcollections)willbestoredbydefault.
Pleasechecktheoutputofthearangod.exeexecutablebeforegoingon.Iftheserverstartedsuccessfully,youshouldseealine ArangoDB
isreadyforbusiness.Havefun!attheendofitsoutput.
Wenowwishtocheckthattheinstallationisworkingcorrectlyandtodothiswewillbeusingtheadministrationwebinterface.Execute
arangod.exeifyouhavenotalreadydoneso,thenopenupyourwebbrowserandpointittothepage:
http://127.0.0.1:8529/
AdvancedStarting
Ifyouwanttoprovideourownstartscripts,youcansettheenvironmentvariableARANGODB_CONFIG_PATH.Thisvariableshould
pointtoadirectorycontainingtheconfigurationfiles.
UsingtheClient
ToconnecttoanalreadyrunningArangoDBserverinstance,thereisashellarangosh.exelocatedin<ROOTDIR>\bin.Thisstartsashell
whichcanbeused–amongstotherthings–toadministerandqueryalocalorremoteArangoDBserver.
Notethatarangosh.exedoesNOTstartaseparateserver,itonlystartstheshell.Touseityoumusthaveaserverrunningsomewhere,
e.g.byusingthearangod.exeexecutable.
arangosh.exeusesconfigurationfromthefilearangosh.conflocatedin<ROOTDIR>\etc\arangodb\.Pleaseadjustthistoyourneedsif
youwanttousedifferentconnectionsettingsetc.
Uninstalling
TouninstalltheArangoserverapplicationyoucanusethewindowscontrolpanel(asyouwouldnormallyuninstallanapplication).Note
however,thatanydatafilescreatedbytheArangoserverwillremainaswellasthe<ROOTDIR>directory.Tocompletethe
uninstallationprocess,removethedatafilesandthe<ROOTDIR>directorymanually.
24
Windows
LimitationsforCygwin
PleasenotesomeimportantlimitationswhenrunningArangoDBunderCygwin:StartingArangoDBcanbestartedfromoutofaCygwin
terminal,butpressingCTRL-Cwillforcefullykilltheserverprocesswithoutgivingitachancetohandlethekillsignal.Inthiscase,a
regularservershutdownisnotpossible,whichmayleaveafileLOCKaroundintheserver'sdatadirectory.Thisfileneedstoberemoved
manuallytomakeArangoDBstartagain.Additionally,asArangoDBdoesnothaveachancetohandlethekillsignal,theservercannot
forcefullyflushanydatatodiskonshutdown,leadingtopotentialdataloss.WhenstartingArangoDBfromaCygwinterminalitmight
alsohappenthatnoerrorsareprintedintheterminaloutput.StartingArangoDBfromanM S-DOScommandpromptdoesnotimpose
theselimitationsandisthusthepreferredmethod.
PleasenotethatArangoDBusesUTF-8asitsinternalencodingandthatthesystemconsolemustsupportaUTF-8codepage(65001)
andfont.ItmaybenecessarytomanuallyswitchtheconsolefonttoafontthatsupportsUTF-8.
25
Compiling
CompilingArangoDBfromscratch
ThefollowingsectionsdescribehowtocompileandbuildtheArangoDBfromscratch.ArangoDBwillcompileonmostLinuxandM ac
OSXsystems.WeassumethatyouusetheGNUC/C++compilerorclang/clang++tocompilethesource.ArangoDBhasbeentested
withthesecompilers,butshouldbeabletocompilewithanyPosix-compliant,C++11-enabledcompiler.Pleaseletusknowwhetheryou
successfullycompileditwithanotherC/C++compiler.
Bydefault,cloningthegithubrepositorywillcheckoutdevel.ThisversioncontainsthedevelopmentversionoftheArangoDB.Usethis
branchifyouwanttomakechangestotheArangoDBsource.
OnWindowsyoufirstneedtoallowandenablesymlinksforyouruser.
PleasecheckoutthecookbookonhowtocompileArangoDB.
26
Authentication
Authentication
ArangoDBallowstorestrictaccesstodatabasestocertainusers.Allusersofthesystemdatabaseareconsideredadministrators.During
installationadefaultuserrootiscreated,whichhasaccesstoalldatabases.
Youshouldcreateadatabaseforyourapplicationtogetherwithauserthathasaccessrightstothisdatabase.SeeM anagingUsers.
Usethearangoshtocreateanewdatabaseanduser.
arangosh>db._createDatabase("example");
arangosh>varusers=require("@arangodb/users");
arangosh>users.save("root@example","password");
arangosh>users.grantDatabase("root@example","example");
Youcannowconnecttothenewdatabaseusingtheuserroot@example.
shell>arangosh--server.username"root@example"--server.databaseexample
27
AccessingtheWebInterface
AccessingtheWebInterface
ArangoDBcomeswithabuilt-inwebinterfaceforadministration.ThewebinterfacecanbeaccessedviatheURL:
http://127.0.0.1:8529
Ifeverythingworksasexpected,youshouldseetheloginview:
FormoreinformationontheArangoDBwebinterface,seeWebInterface
28
ComingfromSQL
ComingfromSQL
Ifyouworkedwitharelationaldatabasemanagementsystem(RDBM S)suchasM ySQL,M ariaDBorPostgreSQL,youwillbefamiliar
withitsquerylanguage,adialectofSQL(StructuredQueryLanguage).
ArangoDB'squerylanguageiscalledAQL.Therearesomesimilaritiesbetweenbothlanguagesdespitethedifferentdatamodelsofthe
databasesystems.ThemostnotabledifferenceisprobablytheconceptofloopsinAQL,whichmakesitfeelmorelikeaprogramming
language.Itsuitestheschema-lessmodelmorenaturalandmakesthequerylanguageverypowerfulwhileremainingeasytoreadand
write.
TogetstartedwithAQL,havealookatourdetailedcomparisonofSQLandAQL.ItwillalsohelpyoutotranslateSQLqueriestoAQL
whenmigratingtoArangoDB.
Howdobrowsevectorstranslateintodocumentqueries?
IntraditionalSQLyoumayeitherfetchallcolumnsofatablerowbyrow,using SELECT*FROMtable,orselectasubsetofthecolumns.
Thelistoftablecolumnstofetchiscommonlycalledcolumnlistorbrowsevector:
SELECTcolumnA,columnB,columnZFROMtable
Sincedocumentsaren'ttwo-dimensional,andneitherdoyouwanttobelimitedtoreturningtwo-dimensionallists,therequirementsfora
querylanguagearehigher.AQListhusalittlebitmorecomplexthanplainSQLatfirst,butoffersmuchmoreflexibilityinthelongrun.It
letsyouhandlearbitrarilystructureddocumentsinconvenientways,mostlyleanedonthesyntaxusedinJavaScript.
Composingthedocumentstobereturned
TheAQL RETURNstatementreturnsoneitemperdocumentitishanded.Youcanreturnthewholedocument,orjustpartsofit.Given
thatoneDocumentisadocument(retrievedlike LEToneDocument=DOCUMENT("myusers/3456789")forinstance),itcanbereturnedas-is
likethis:
RETURNoneDocument
[
{
"_id":"myusers/3456789",
"_key":"3456789"
"_rev":"14253647",
"firstName":"John",
"lastName":"Doe",
"address":{
"city":"Gotham",
"street":"RoadToNowhere1"
},
"hobbies":[
{name:"swimming",howFavorite:10},
{name:"biking",howFavorite:6},
{name:"programming",howFavorite:4}
]
}
]
Returnthehobbiessub-structureonly:
RETURNoneDocument.hobbies
[
[
{name:"swimming",howFavorite:10},
{name:"biking",howFavorite:6},
{name:"programming",howFavorite:4}
29
ComingfromSQL
]
]
Returnthehobbiesandtheaddress:
RETURN{
hobbies:oneDocument.hobbies,
address:oneDocument.address
}
[
{
hobbies:[
{name:"swimming",howFavorite:10},
{name:"biking",howFavorite:6},
{name:"programming",howFavorite:4}
],
address:{
"city":"Gotham",
"street":"RoadToNowhere1"
}
}
]
Returnthefirsthobbyonly:
RETURNoneDocument.hobbies[0].name
[
"swimming"
]
Returnalistofallhobbystrings:
RETURN{hobbies:oneDocument.hobbies[*].name}
[
{hobbies:["swimming","biking","porgramming"]}
]
M orecomplexarrayandobjectmanipulationscanbedoneusingAQLfunctionsandoperators.
30
Highlights
Highlights
Version3.2
RocksDBS torageEngine:YoucannowuseasmuchdatainArangoDBasyoucanfitonyourdisk.Plus,youcanenjoy
performanceboostsonwritesbyhavingonlydocument-levellocks
Pregel:WeimplementeddistributedgraphprocessingwithPregeltodiscoverhiddenpatterns,identifycommunitiesandperformindepthanalyticsoflargegraphdatasets.
Fault-TolerantFoxx:TheFoxxmanagementinternalshavebeenrewrittenfromthegrounduptomakesuremulti-coordinator
clustersetupsalwayskeeptheirservicesinsyncandnewcoordinatorsarefullyinitializedevenwhenallexistingcoordinatorsare
unavailable.
Enterprise:Workingwithsomeofourlargestcustomers,we’veaddedfurthersecurityandscalabilityfeaturestoArangoDB
EnterpriselikeLDAPintegration,EncryptionatRest,andthebrandnewSatelliteCollections.
AlsoseeWhat'sNewin3.2.
Version3.1
S martGraphs:Scalewithgraphstoaclusterandstayperformant.WithSmartGraphsyoucanusethe"smartness"ofyour
applicationlayertoshardyourgraphefficientlytoyourmachinesandlettraversalsrunlocally.
EncryptionControl:ChooseyourlevelofSSLencryption
Auditing:KeepadetailedlogofalltheimportantthingsthathappenedinArangoDB.
AlsoseeWhat'sNewin3.1.
Version3.0
self-organizingclusterwithsynchronousreplication,master/mastersetup,sharednothingarchitecture,clustermanagement
agency.
Deeplyintegrated,nativeAQLgraphtraversal
VelocyPackasnewinternalbinarystorageformataswellasforintermediateAQLvalues.
PersistentindexesviaRocksDBsuitableforsortingandrangequeries.
Foxx3.0:overhauledJSframeworkfordata-centricmicroservices
SignificantlyimprovedWebInterface
AlsoseeWhat'sNewin3.0.
31
Scalability
Scalability
ArangoDBisadistributeddatabasesupportingmultipledatamodels,andcanthusbescaledhorizontally,thatis,byusingmanyservers,
typicallybasedoncommodityhardware.Thisapproachnotonlydeliversperformanceaswellascapacityimprovements,butalso
achievesresiliencebymeansofreplicationandautomaticfail-over.Furthermore,onecanbuildsystemsthatscaletheircapacity
dynamicallyupanddownautomaticallyaccordingtodemand.
OnecanalsoscaleArangoDBvertically,thatis,byusingeverlargerservers.ThereisnobuiltinlimitationinArangoDB,forexample,the
serverwillautomaticallyusemorethreadsifmoreCPUsarepresent.
However,scalingverticallyhasthedisadvantagethatthecostsgrowfasterthanlinearwiththesizeoftheserver,andnoneofthe
resilienceanddynamicalcapabilitiescanbeachievedinthisway.
InthischapterweexplainthedistributedarchitectureofArangoDBanddiscussitsscalabilityfeaturesandlimitations:
ArangoDB'sdistributedarchitecture
Differentdatamodelsandscalability
Limitations
32
Architecture
Architecture
TheclusterarchitectureofArangoDBisaCPmaster/mastermodelwithnosinglepointoffailure.With"CP"wemeanthatinthe
presenceofanetworkpartition,thedatabaseprefersinternalconsistencyoveravailability.With"master/master"wemeanthatclients
cansendtheirrequeststoanarbitrarynode,andexperiencethesameviewonthedatabaseregardless."Nosinglepointoffailure"means
thattheclustercancontinuetoserverequests,evenifonemachinefailscompletely.
Inthisway,ArangoDBhasbeendesignedasadistributedmulti-modeldatabase.Thissectiongivesashortoutlineonthecluster
architectureandhowtheabovefeaturesandcapabilitiesareachieved.
StructureofanArangoDBcluster
AnArangoDBclusterconsistsofanumberofArangoDBinstanceswhichtalktoeachotheroverthenetwork.Theyplaydifferentroles,
whichwillbeexplainedindetailbelow.Thecurrentconfigurationoftheclusterisheldinthe"Agency",whichisahighly-available
resilientkey/valuestorebasedonanoddnumberofArangoDBinstancesrunningRaftConsensusProtocol.
ForthevariousinstancesinanArangoDBclusterthereare4distinctroles:Agents,Coordinators,PrimaryandSecondaryDBservers.In
thefollowingsectionswewillshedlightoneachofthem.NotethatthetasksforallrolesrunthesamebinaryfromthesameDocker
image.
Agents
OneormultipleAgentsformtheAgencyinanArangoDBcluster.TheAgencyisthecentralplacetostoretheconfigurationinacluster.
Itperformsleaderelectionsandprovidesothersynchronizationservicesforthewholecluster.WithouttheAgencynoneoftheother
componentscanoperate.
Whilegenerallyinvisibletotheoutsideitistheheartofthecluster.Assuch,faulttoleranceisofcourseamusthavefortheAgency.To
achievethattheAgentsareusingtheRaftConsensusAlgorithm.Thealgorithmformallyguaranteesconflictfreeconfiguration
managementwithintheArangoDBcluster.
AtitscoretheAgencymanagesabigconfigurationtree.Itsupportstransactionalreadandwriteoperationsonthistree,andotherservers
cansubscribetoHTTPcallbacksforallchangestothetree.
Coordinators
Coordinatorsshouldbeaccessiblefromtheoutside.Thesearetheonestheclientstalkto.Theywillcoordinateclustertaskslike
executingqueriesandrunningFoxxservices.Theyknowwherethedataisstoredandwilloptimizewheretorunusersuppliedqueriesor
partsthereof.Coordinatorsarestatelessandcanthuseasilybeshutdownandrestartedasneeded.
PrimaryDBservers
PrimaryDBserversaretheoneswherethedataisactuallyhosted.Theyhostshardsofdataandusingsynchronousreplicationaprimary
mayeitherbeleaderorfollowerforashard.
Theyshouldnotbeaccessedfromtheoutsidebutindirectlythroughthecoordinators.Theymayalsoexecutequeriesinpartorasa
wholewhenaskedbyacoordinator.
Secondaries
SecondaryDBserversareasynchronousreplicasofprimaries.Ifoneisusingonlysynchronousreplication,onedoesnotneedsecondaries
atall.Foreachprimary,therecanbeoneormoresecondaries.Sincethereplicationworksasynchronously(eventualconsistency),the
replicationdoesnotimpedetheperformanceoftheprimaries.Ontheotherhand,theirreplicaofthedatacanbeslightlyoutofdate.The
secondariesareperfectlysuitableforbackupsastheydon'tinterferewiththenormalclusteroperation.
ClusterID
Everynon-AgencyArangoDBinstanceinaclusterisassignedauniqueIDduringitsstartup.UsingitsIDanodeisidentifiable
throughoutthecluster.AllclusteroperationswillcommunicateviathisID.
33
Architecture
Sharding
UsingtherolesoutlinedaboveanArangoDBclusterisabletodistributedatainsocalledshardsacrossmultipleprimaries.Fromthe
outsidethisprocessisfullytransparentandassuchweachievethegoalsofwhatothersystemscall"master-masterreplication".Inan
ArangoDBclusteryoutalktoanycoordinatorandwheneveryoureadorwritedataitwillautomaticallyfigureoutwherethedatais
stored(read)ortobestored(write).TheinformationabouttheshardsissharedacrossthecoordinatorsusingtheAgency.
AlsoseeShardingintheAdministrationchapter.
Manysensibleconfigurations
Thisarchitectureisveryflexibleandthusallowsmanyconfigurations,whicharesuitablefordifferentusagescenarios:
1. ThedefaultconfigurationistorunexactlyonecoordinatorandoneprimaryDBserveroneachmachine.Thisachievestheclassical
master/mastersetup,sincethereisaperfectsymmetrybetweenthedifferentnodes,clientscanequallywelltalktoanyoneofthe
coordinatorsandallexposethesameviewtothedatastore.
2. OnecandeploymorecoordinatorsthanDBservers.ThisisasensibleapproachifoneneedsalotofCPUpowerfortheFoxx
services,becausetheyrunonthecoordinators.
3. OnecandeploymoreDBserversthancoordinatorsifmoredatacapacityisneededandthequeryperformanceisthelesser
bottleneck
4. Onecandeployacoordinatoroneachmachinewhereanapplicationserver(e.g.anode.jsserver)runs,andtheAgentsand
DBserversonaseparatesetofmachineselsewhere.Thisavoidsanetworkhopbetweentheapplicationserverandthedatabaseand
thusdecreaseslatency.Essentially,thismovessomeofthedatabasedistributionlogictothemachinewheretheclientruns.
Theseforshallsufficefornow.Theimportantpieceofinformationhereisthatthecoordinatorlayercanbescaledanddeployed
independentlyfromtheDBserverlayer.
Replication
ArangoDBofferstwowaysofdatareplicationwithinacluster,synchronousandasynchronous.Inthissectionweexplainsomedetails
andhighlighttheadvantagesanddisadvantagesrespectively.
Synchronousreplicationwithautomaticfail-over
Synchronousreplicationworksonaper-shardbasis.Oneconfiguresforeachcollection,howmanycopiesofeachshardarekeptinthe
cluster.Atanygiventime,oneofthecopiesisdeclaredtobethe"leader"andallotherreplicasare"followers".Writeoperationsforthis
shardarealwayssenttotheDBserverwhichhappenstoholdtheleadercopy,whichinturnreplicatesthechangestoallfollowersbefore
theoperationisconsideredtobedoneandreportedbacktothecoordinator.Readoperationsareallservedbytheserverholdingthe
leadercopy,thisallowstoprovidesnapshotsemanticsforcomplextransactions.
IfaDBserverfailsthatholdsafollowercopyofashard,thentheleadercannolongersynchronizeitschangestothatfollower.Aftera
shorttimeout(3seconds),theleadergivesuponthefollower,declaresittobeoutofsync,andcontinuesservicewithoutthefollower.
Whentheserverwiththefollowercopycomesback,itautomaticallyresynchronizesitsdatawiththeleaderandsynchronousreplication
isrestored.
IfaDBserverfailsthatholdsaleadercopyofashard,thentheleadercannolongerserveanyrequests.Itwillnolongersendaheartbeat
totheAgency.Therefore,asupervisionprocessrunningintheRaftleaderoftheAgency,cantakethenecessaryaction(after15seconds
ofmissingheartbeats),namelytopromoteoneoftheserversthatholdin-syncreplicasoftheshardtoleaderforthatshard.Thisinvolves
areconfigurationintheAgencyandleadstothefactthatcoordinatorsnowcontactadifferentDBserverforrequeststothisshard.Service
resumes.Theothersurvivingreplicasautomaticallyresynchronizetheirdatawiththenewleader.WhentheDBserverwiththeoriginal
leadercopycomesback,itnoticesthatitnowholdsafollowerreplica,resynchronizesitsdatawiththenewleaderandorderisrestored.
Allsharddatasynchronizationsaredoneinanincrementalway,suchthatresynchronizationsarequick.Thistechnologyallowstomove
shards(followerandleaderones)betweenDBserverswithoutserviceinterruptions.Therefore,anArangoDBclustercanmoveallthe
dataonaspecificDBservertootherDBserversandthenshutdownthatserverinacontrolledway.Thisallowstoscaledownan
ArangoDBclusterwithoutserviceinterruption,lossoffaulttoleranceordataloss.Furthermore,onecanre-balancethedistributionofthe
shards,eithermanuallyorautomatically.
AlltheseoperationscanbetriggeredviaaREST/JSONAPIorviathegraphicalwebUI.Allfail-overoperationsarecompletelyhandled
withintheArangoDBcluster.
34
Architecture
Obviously,synchronousreplicationinvolvesacertainincreasedlatencyforwriteoperations,simplybecausethereisonemorenetwork
hopwithintheclusterforeveryrequest.Thereforetheusercansetthereplicationfactorto1,whichmeansthatonlyonecopyofeach
shardiskept,therebyswitchingoffsynchronousreplication.Thisisasuitablesettingforlessimportantoreasilyrecoverabledatafor
whichlowlatencywriteoperationsmatter.
Asynchronousreplicationwithautomaticfail-over
Asynchronousreplicationworksdifferently,inthatitisorganizedusingprimaryandsecondaryDBservers.Eachsecondaryserver
replicatesallthedataheldonaprimarybypollinginanasynchronousway.Thisprocesshasverylittleimpactontheperformanceofthe
primary.Thedisadvantageisthatthereisadelaybetweentheconfirmationofawriteoperationthatissenttotheclientandtheactual
replicationofthedata.Ifthemasterserverfailsduringthisdelay,thencommittedandconfirmeddatacanbelost.
Nevertheless,wealsoofferautomaticfail-overwiththissetup.Contrarytothesynchronouscase,herethefail-overmanagementisdone
fromoutsidetheArangoDBcluster.InafutureversionwemightmovethismanagementintothesupervisionprocessintheAgency,but
asofnow,themanagementisdoneviatheM esosframeworkschedulerforArangoDB(seebelow).
ThegranularityofthereplicationisawholeArangoDBinstancewithalldatathatresidesonthatinstance,whichmeansthatyouneed
twiceasmanyinstancesaswithoutasynchronousreplication.Synchronousreplicationismoreflexibleinthatrespect,youcanhave
smallerandlargerinstances,andifonefails,thedatacanberebalancedacrosstheremainingones.
Microservicesandzeroadministation
ThedesignandcapabilitiesofArangoDBaregearedtowardsusageinmodernmicroservicearchitecturesofapplications.WiththeFoxx
servicesitisveryeasytodeployadatacentricmicroservicewithinanArangoDBcluster.
Inaddition,onecandeploymultipleinstancesofArangoDBwithinthesameproject.Onepartoftheprojectmightneedascalable
documentstore,anothermightneedagraphdatabase,andyetanothermightneedthefullpowerofamulti-modeldatabaseactually
mixingthevariousdatamodels.Thereareenormousefficiencybenefitstobereapedbybeingabletouseasingletechnologyforvarious
rolesinaproject.
TosimplifyliveofthedevopsinsuchascenariowetryasmuchaspossibletouseazeroadministrationapproachforArangoDB.A
runningArangoDBclusterisresilientagainstfailuresandessentiallyrepairsitselfincaseoftemporaryfailures.Seethenextsectionfor
furthercapabilitiesinthisdirection.
ApacheMesosintegration
Forthedistributedsetup,weusetheApacheM esosinfrastructurebydefault.ArangoDBisafullycertifiedpackageforDC/OSandcan
thusbedeployedessentiallywithafewmouseclicksorasinglecommand,onceyouhaveanexistingDC/OScluster.Butevenonaplain
ApacheM esosclusteronecandeployArangoDBviaM arathonwithasingleAPIcallandsomeJSONconfiguration.
Theadvantageofthisapproachisthatwecannotonlyimplementtheinitialdeployment,butalsothelatermanagementofautomatic
replacementoffailedinstancesandthescalingoftheArangoDBcluster(triggeredmanuallyorevenautomatically).Sinceall
manipulationsareeitherviathegraphicalwebUIorviaJSON/RESTcalls,onecanevenimplementauto-scalingveryeasily.
ADC/OSclusterisaverynaturalenvironmenttodeploymicroservicearchitectures,sinceitissoconvenienttodeployvariousservices,
includingpotentiallymultipleArangoDBclusterinstanceswithinthesameDC/OScluster.Thebuilt-inservicediscoverymakesit
extremelysimpletoconnectthevariousmicroservicesandM esosautomaticallytakescareofthedistributionanddeploymentofthe
varioustasks.
AsofJune2016,weofferApacheM esosintegration,latertherewillbeintegrationwithotherclustermanagementinfrastructures.See
theDeploymentchapteranditssubsectionsforinstructions.
ItispossibletodeployanArangoDBclusterbysimplylaunchingabunchofDockercontainerswiththerightcommandlineoptionsto
linkthemup,orevenonasinglemachinestartingmultipleArangoDBprocesses.Inthatcase,synchronousreplicationwillworkwithin
thedeployedArangoDBcluster,andautomaticfail-overinthesensethatthedutiesofafailedserverwillautomaticallybeassignedto
another,survivingone.However,sincetheArangoDBclustercannotwithinitselflaunchadditionalinstances,replacementoffailednodes
isnotautomaticandscalingupanddownhastobemanagedmanually.Thisiswhywedonotrecommendthissetupforproduction
deployment.
Authentication
35
Architecture
Asofversion3.0ArangoDBauthenticationisNOTsupportedwithinacluster.YouHAVEtoproperlysecureyourclustertothe
outside.M ostsetupswillhaveasecureddatacenteranywayandArangoDBwillbeaccessedfromtheoutsideviaanapplication.Tothis
applicationonlythecoordinatorsneedtobemadeavailable.Ifyouwanttoisolateevenfurtheryoucaninstallareverseproxylike
haproxyornginxinfrontofthecoordinators(thatwillalsoalloweasyaccessfromtheapplication).
Authenticationintheclusterwillbeaddedsoonaftertheinitial3.0release.
36
Datamodels
Differentdatamodelsandscalability
InthissectionwediscussscalabilityinthecontextofthedifferentdatamodelssupportedbyArangoDB.
Key/valuepairs
Thekey/valuestoredatamodelistheeasiesttoscale.InArangoDB,thisisimplementedinthesensethatadocumentcollectionalways
hasaprimarykey _keyattributeandintheabsenceoffurthersecondaryindexesthedocumentcollectionbehaveslikeasimple
key/valuestore.
Theonlyoperationsthatarepossibleinthiscontextaresinglekeylookupsandkey/valuepairinsertionsandupdates.If _keyisthe
onlyshardingattributethentheshardingisdonewithrespecttotheprimarykeyandalltheseoperationsscalelinearly.Iftheshardingis
doneusingdifferentshardkeys,thenalookupofasinglekeyinvolvesaskingallshardsandthusdoesnotscalelinearly.
Documentstore
Forthedocumentstorecaseeveninthepresenceofsecondaryindexesessentiallythesameargumentsapply,sinceanindexforasharded
collectionissimplythesameasalocalindexforeachshard.Therefore,singledocumentoperationsstillscalelinearlywiththesizeofthe
cluster,unlessaspecialshardingconfigurationmakeslookupsorwriteoperationsmoreexpensive.
ForadeeperanalysisofthistopicseethisblogpostinwhichgoodlinearscalabilityofArangoDBforsingledocumentoperationsis
demonstrated.
Complexqueriesandjoins
TheAQLquerylanguageallowscomplexqueries,usingmultiplecollections,secondaryindexesaswellasjoins.Inparticularwiththe
latter,scalingcanbeachallenge,sinceifthedatatobejoinedresidesondifferentmachines,alotofcommunicationhastohappen.The
AQLqueryexecutionengineorganizesadatapipelineacrosstheclustertoputtogethertheresultsinthemostefficientway.Thequery
optimizerisawareoftheclusterstructureandknowswhatdataiswhereandhowitisindexed.Therefore,itcanarriveataninformed
decisionaboutwhatpartsofthequeryoughttorunwhereinthecluster.
Nevertheless,forcertaincomplicatedjoins,therearelimitsastowhatcanbeachieved.
Graphdatabase
Graphdatabasesareparticularlygoodatqueriesongraphsthatinvolvepathsinthegraphofanaprioriunknownlength.Forexample,
findingtheshortestpathbetweentwoverticesinagraph,orfindingallpathsthatmatchacertainpatternstartingatagivenvertexare
suchexamples.
However,iftheverticesandedgesalongtheoccurringpathsaredistributedacrossthecluster,thenalotofcommunicationisnecessary
betweennodes,andperformancesuffers.Toachievegoodperformanceatscale,itisthereforenecessarytogetthedistributionofthe
graphdataacrosstheshardsintheclusterright.M ostofthetime,theapplicationdevelopersandusersofArangoDBknowbest,how
theirgraphsarastructured.Therefore,ArangoDBallowsuserstospecify,accordingtowhichattributesthegraphdataissharded.A
usefulfirststepisusuallytomakesurethattheedgesoriginatingatavertexresideonthesameclusternodeasthevertex.
37
Limitations
Limitations
ArangoDBhasnobuilt-inlimitationstohorizontalscalability.ThecentralresilientAgencywilleasilysustainhundredsofDBserversand
coordinators,andtheusualdatabaseoperationsworkcompletelydecentrallyanddonotrequireassistanceoftheAgency.
Likewise,thesupervisionprocessintheAgencycaneasilydealwithlotsofservers,sinceallitsactivitiesarenotperformancecritical.
Obviously,anArangoDBclusterislimitedbytheavailableresourcesofCPU,memory,diskandnetworkbandwidthandlatency.
38
Datamodels&modeling
Datamodels&modeling
ThischapterintroducesArangoDB'scoreconceptsandcovers
itsdatamodel(ordatamodelsrespectively),
theterminologyusedthroughoutthedatabasesystemandinthisdocumentation,aswellas
aspectstoconsiderwhenmodelingyourdatatostrikeabalancebetweennaturaldatastructuresandgreatperformance
Youwillalsofindusageexamplesonhowtointeractwiththedatabasesystemusingarangosh,e.g.howtocreateanddropdatabases/
collections,orhowtosave,update,replaceandremovedocuments.Youcandoallthisusingthewebinterfaceaswellandmaytherefore
skipthesesectionsasbeginner.
39
Concepts
Concepts
DatabaseInteraction
ArangoDBisadatabasethatservesdocumentstoclients.ThesedocumentsaretransportedusingJSONviaaTCPconnection,usingthe
HTTPprotocol.ARESTAPIisprovidedtointeractwiththedatabasesystem.
ThewebinterfacethatcomeswithArangoDB,calledAardvark,providesgraphicaluserinterfacethatiseasytouse.Aninteractiveshell,
calledArangosh,isalsoshipped.Inaddition,therearesocalleddriversthatmakeiteasytousethedatabasesysteminvarious
environmentsandprogramminglanguages.AllthesetoolsusetheHTTPinterfaceoftheserverandremovethenecessitytorollownlowlevelcodeforbasiccommunicationinmostcases.
Datamodel
ThedocumentsyoucanstoreinArangoDBcloselyfollowtheJSONformat,althoughtheyarestoredinabinaryformatcalled
VelocyPack.Adocumentcontainszeroormoreattributes,eachoftheseattributeshavingavalue.Avaluecaneitherbeanatomictype,i.
e.number,string,booleanornull,oracompoundtype,i.e.anarrayorembeddeddocument/object.Arraysandsub-objectscancontain
allofthesetypes,whichmeansthatarbitrarilynesteddatastructurescanberepresentedinasingledocument.
Documentsaregroupedintocollections.Acollectioncontainszeroormoredocuments.Ifyouarefamiliarwithrelationaldatabase
managementsystems(RDBM S)thenitissafetocomparecollectionstotablesanddocumentstorows.Thedifferenceisthatina
traditionalRDBM S,youhavetodefinecolumnsbeforeyoucanstorerecordsinatable.Suchdefinitionsarealsoknownasschemas.
ArangoDBisschema-less,whichmeansthatthereisnoneedtodefinewhatattributesadocumentcanhave.Everysingledocumentcan
haveacompletelydifferentstructureandstillbestoredtogetherwithotherdocumentsinasinglecollection.Inpractice,therewillbe
commondenominatorsamongthedocumentsinacollection,butthedatabasesystemitselfdoesn'tforceyoutolimityourselftoacertain
datastructure.
Therearetwotypesofcollections:documentcollection(alsoreferedtoasvertexcollectionsinthecontextofgraphs)aswellasedge
collections.Edgecollectionsstoredocumentsaswell,buttheyincludetwospecialattributes,_fromand_to,whichareusedtocreate
relationsbetweendocuments.Usually,twodocuments(vertices)storedindocumentcollectionsarelinkedbyadocument(edge)stored
inanedgecollection.ThisisArangoDB'sgraphdatamodel.Itfollowsthemathematicalconceptofadirected,labeledgraph,exceptthat
edgesdon'tjusthavelabels,butarefull-blowndocuments.
Collectionsexistinsideofdatabases.Therecanbeoneormanydatabases.Differentdatabasesareusuallyusedformultitenantsetups,
asthedatainsidethem(collections,documentsetc.)isisolatedfromoneanother.Thedefaultdatabase_systemisspecial,becauseit
cannotberemoved.Databaseusersaremanagedinthisdatabase,andtheircredentialsarevalidforalldatabasesofaserverinstance.
DataRetrieval
Queriesareusedtofilterdocumentsbasedoncertaincriteria,tocomputenewdata,aswellastomanipulateordeleteexisting
documents.Queriescanbeassimpleasa"querybyexample"orascomplexas"joins"usingmanycollectionsortraversinggraph
structures.TheyarewrittenintheArangoDBQueryLanguage(AQL).
Cursorsareusedtoiterateovertheresultofqueries,sothatyougeteasilyprocessablebatchesinsteadofonebighunk.
Indexesareusedtospeedupsearches.Therearevarioustypesofindexes,suchashashindexesandgeoindexes.
40
Databases
HandlingDatabases
ThisisanintroductiontomanagingdatabasesinArangoDBfromwithinJavaScript.
WhenyouhaveanestablishedconnectiontoArangoDB,thecurrentdatabasecanbechangedexplicitlyusingthedb._useDatabase()
method.Thiswillswitchtothespecifieddatabase(provideditexistsandtheusercanconnecttoit).Fromthispointon,anyfollowing
actioninthesameshellorconnectionwillusethespecifieddatabase,unlessotherwisespecified.
Note:Ifthedatabaseischanged,clientdriversneedtostorethecurrentdatabasenameontheirside,too.Thisisbecauseconnectionsin
ArangoDBdonotcontainanystateinformation.AllstateinformationiscontainedintheHTTPrequest/responsedata.
Toconnecttoaspecificdatabaseafterarangoshhasstartedusethecommanddescribedabove.Itisalsopossibletospecifyadatabase
namewheninvokingarangosh.Forthispurpose,usethecommand-lineparameter--server.database,e.g.
>arangosh--server.databasetest
Pleasenotethatcommands,actions,scriptsorAQLqueriesshouldneveraccessmultipledatabases,eveniftheyexist.Theonlyintended
andsupportedwayinArangoDBistouseonedatabaseatatimeforacommand,anaction,ascriptoraquery.Operationsstartedinone
databasemustnotswitchthedatabaselaterandcontinueoperatinginanother.
41
WorkingwithDatabases
WorkingwithDatabases
DatabaseMethods
ThefollowingmethodsareavailabletomanagedatabasesviaJavaScript.Pleasenotethatseveralofthesemethodscanbeusedfromthe
_systemdatabaseonly.
Name
returnthedatabasename db._name()
Returnsthenameofthecurrentdatabaseasastring.
Examples
arangosh>require("@arangodb").db._name();
_system
ID
returnthedatabaseid db._id()
Returnstheidofthecurrentdatabaseasastring.
Examples
arangosh>require("@arangodb").db._id();
1
Path
returnthepathtodatabasefiles db._path()
Returnsthefilesystempathofthecurrentdatabaseasastring.
Examples
arangosh>require("@arangodb").db._path();
/var/folders/sb/5_j9t5wx4d7bpprydjdb0xtc0000gn/T/arangosh_RPN6DS/tmp-26715947380293/data/databases/database-1
isSystem
returnthedatabasetype db._isSystem()
Returnswhetherthecurrentlyuseddatabaseisthe_systemdatabase.Thesystemdatabasehassomespecialprivilegesandproperties,for
example,databasemanagementoperationssuchascreateordropcanonlybeexecutedfromwithinthisdatabase.Additionally,the
_systemdatabaseitselfcannotbedropped.
UseDatabase
changethecurrentdatabase db._useDatabase(name)
Changesthecurrentdatabasetothedatabasespecifiedbyname.Notethatthedatabasespecifiedbynamemustalreadyexist.
Changingthedatabasemightbedisallowedinsomecontexts,forexampleserver-sideactions(includingFoxx).
42
WorkingwithDatabases
Whenperformingthiscommandfromarangosh,thecurrentcredentials(usernameandpassword)willbere-used.Thesecredentialsmight
notbevalidtoconnecttothedatabasespecifiedbyname.Additionally,thedatabaseonlybeaccessedfromcertainendpointsonly.In
thiscase,switchingthedatabasemightnotwork,andtheconnection/sessionshouldbeclosedandrestartedwithdifferentusernameand
passwordcredentialsand/orendpointdata.
ListDatabases
returnthelistofallexistingdatabases db._databases()
Returnsthelistofalldatabases.Thismethodcanonlybeusedfromwithinthe_systemdatabase.
CreateDatabase
createanewdatabase db._createDatabase(name,options,users)
Createsanewdatabasewiththenamespecifiedbyname.Therearerestrictionsfordatabasenames(seeDatabaseNames).
Notethatevenifthedatabaseiscreatedsuccessfully,therewillbenochangeintothecurrentdatabasetothenewdatabase.Changingthe
currentdatabasemustexplicitlyberequestedbyusingthedb._useDatabasemethod.
Theoptionsattributecurrentlyhasnomeaningandisreservedforfutureuse.
Theoptionalusersattributecanbeusedtocreateinitialusersforthenewdatabase.Ifspecified,itmustbealistofuserobjects.Each
userobjectcancontainthefollowingattributes:
username:theusernameasastring.Thisattributeismandatory.
passwd:theuserpasswordasastring.Ifnotspecified,thenitdefaultstoanemptystring.
active:abooleanflagindicatingwhethertheuseraccountshouldbeactiveornot.Thedefaultvalueistrue.
extra:anoptionalJSONobjectwithextrauserinformation.Thedatacontainedinextrawillbestoredfortheuserbutnotbe
interpretedfurtherbyArangoDB.
Ifnoinitialusersarespecified,adefaultuserrootwillbecreatedwithanemptystringpassword.Thisensuresthatthenewdatabasewill
beaccessibleviaHTTPafteritiscreated.
Youcancreateusersinadatabaseifnoinitialuserisspecified.Switchintothenewdatabase(usernameandpasswordmustbeidentical
tothecurrentsession)andaddormodifyuserswiththefollowingcommands.
require("@arangodb/users").save(username,password,true);
require("@arangodb/users").update(username,password,true);
require("@arangodb/users").remove(username);
Alternatively,youcanspecifyuserdatadirectly.Forexample:
db._createDatabase("newDB",{},[{username:"newUser",passwd:"123456",active:true}])
Thosemethodscanonlybeusedfromwithinthe_systemdatabase.
DropDatabase
dropanexistingdatabase db._dropDatabase(name)
Dropsthedatabasespecifiedbyname.Thedatabasespecifiedbynamemustexist.
Note:Droppingdatabasesisonlypossiblefromwithinthe_systemdatabase.The_systemdatabaseitselfcannotbedropped.
Databasesaredroppedasynchronously,andwillbephysicallyremovedifallclientshavedisconnectedandreferenceshavebeengarbagecollected.
Enginestatistics
retrievestatisticsrelatedtothestorageengine-rocksdb db._engineStats()
Returnssomestatisticsrelatedtostorageengineactivity,includingfiguresaboutdatasize,cacheusage,etc.
43
WorkingwithDatabases
Note:CurrentlythisonlyproducesusefuloutputfortheRocksDBengine.
44
NotesaboutDatabases
NotesaboutDatabases
Pleasekeepinmindthateachdatabasecontainsitsownsystemcollections,whichneedtobesetupwhenadatabaseiscreated.Thiswill
makethecreationofadatabasetakeawhile.
Replicationisconfiguredonaper-databaselevel,meaningthatanyreplicationloggingorapplyingforanewdatabasemustbeconfigured
explicitlyafteranewdatabasehasbeencreated.
Foxxapplicationsarealsoavailableonlyinthecontextofthedatabasetheyhavebeeninstalledin.Anewdatabasewillonlyprovide
accesstothesystemapplicationsshippedwithArangoDB(thatisthewebinterfaceatthemoment)andnootherFoxxapplicationsuntil
theyareexplicitlyinstalledfortheparticulardatabase.
45
Collections
JavaScriptInterfacetoCollections
ThisisanintroductiontoArangoDB'sinterfaceforcollectionsandhowtohandlecollectionsfromtheJavaScriptshellarangosh.For
otherlanguagesseethecorrespondinglanguageAPI.
Themostimportantcallisthecalltocreateanewcollection.
AddressofaCollection
AllcollectionsinArangoDBhaveauniqueidentifierandauniquename.ArangoDBinternallyusesthecollection'suniqueidentifierto
lookupcollections.Thisidentifier,however,ismanagedbyArangoDBandtheuserhasnocontroloverit.Inordertoallowuserstouse
theirownnames,eachcollectionalsohasauniquenamewhichisspecifiedbytheuser.Toaccessacollectionfromtheuserperspective,
thecollectionnameshouldbeused,i.e.:
Collection
db._collection(collection-name)
Acollectioniscreatedbya"db._create"call.
Forexample:Assumethatthecollectionidentifieris7254820andthenameisdemo,thenthecollectioncanbeaccessedas:
db._collection("demo")
Ifnocollectionwithsuchanameexists,thennullisreturned.
Thereisashort-cutthatcanbeusedfornon-systemcollections:
Collectionname
db.collection-name
Thiscallwilleitherreturnthecollectionnameddb.collection-nameorcreateanewonewiththatnameandasetofdefaultproperties.
Note:Creatingacollectionontheflyusingdb.collection-nameisnotrecommendanddoesnotworkinarangosh.Tocreateanew
collection,pleaseuse
Create
db._create(collection-name)
Thiscallwillcreateanewcollectioncalledcollection-name.ThismethodisadatabasemethodandisdocumentedindetailatDatabase
M ethods
Synchronousreplication
StartinginArangoDB3.0,thedistributedversionofferssynchronousreplication,whichmeansthatthereistheoptiontoreplicateall
dataautomaticallywithintheArangoDBcluster.Thisisconfiguredforshardedcollectionsonapercollectionbasisbyspecifyinga
"replicationfactor"whenthecollectioniscreated.Areplicationfactorofkmeansthataltogetherkcopiesofeachshardarekeptinthe
clusteronkdifferentservers,andarekeptinsync.Thatis,everywriteoperationisautomaticallyreplicatedonallcopies.
Thisisorganisedusingaleader/followermodel.Atalltimes,oneoftheserversholdingreplicasforashardis"theleader"andallothers
are"followers",thisconfigurationisheldintheAgency(seeScalabilityfordetailsoftheArangoDBclusterarchitecture).Everywrite
operationissenttotheleaderbyoneofthecoordinators,andthenreplicatedtoallfollowersbeforetheoperationisreportedtohave
succeeded.Theleaderkeepsarecordofwhichfollowersarecurrentlyinsync.Incaseofnetworkproblemsorafailureofafollower,a
leadercanandwilldropafollowertemporarilyafter3seconds,suchthatservicecanresume.Induecourse,thefollowerwill
automaticallyresynchronizewiththeleadertorestoreresilience.
46
Collections
Ifaleaderfails,theclusterAgencyautomaticallyinitiatesafailoverroutineafteraround15seconds,promotingoneofthefollowersto
leader.Theotherfollowers(andtheformerleader,whenitcomesback),automaticallyresynchronizewiththenewleadertorestore
resilience.Usually,thiswholefailoverprocedurecanbehandledtransparentlyforthecoordinator,suchthattheusercodedoesnoteven
seeanerrormessage.
Obviously,thisfaulttolerancecomesatacostofincreasedlatency.Eachwriteoperationneedsanadditionalnetworkroundtripforthe
synchronousreplicationofthefollowers,butallreplicationoperationstoallfollowershappenconcurrently.Thisis,whythedefault
replicationfactoris1,whichmeansnoreplication.
Fordetailsonhowtoswitchonsynchronousreplicationforacollection,seethedatabasemethod db._create(collection-name)inthe
sectionaboutDatabaseM ethods.
47
CollectionM ethods
CollectionMethods
Drop
dropsacollection collection.drop(options)
Dropsacollectionandallitsindexesanddata.Inordertodropasystemcollection,anoptionsobjectwithattributeisSystemsettotrue
mustbespecified.
Note:droppingacollectioninacluster,whichisprototypeforsharinginothercollectionsisprohibited.Inordertobeabletodropsuch
acollection,alldependentcollectionsmustbedroppedfirst.
Examples
arangosh>col=db.example;
[ArangoCollection16346,"example"(typedocument,statusloaded)]
arangosh>col.drop();
arangosh>col;
[ArangoCollection16346,"example"(typedocument,statusdeleted)]
arangosh>col=db._example;
[ArangoCollection16356,"_example"(typedocument,statusloaded)]
arangosh>col.drop({isSystem:true});
arangosh>col;
[ArangoCollection16356,"_example"(typedocument,statusdeleted)]
Truncate
truncatesacollection collection.truncate()
Truncatesacollection,removingalldocumentsbutkeepingallitsindexes.
Examples
Truncatesacollection:
arangosh>col=db.example;
arangosh>col.save({"Hello":"World"});
arangosh>col.count();
arangosh>col.truncate();
arangosh>col.count();
showexecutionresults
Properties
getsorsetsthepropertiesofacollection collection.properties()Returnsanobjectcontainingallcollectionproperties.
waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheM M Filesstorageengineonly.
isVolatile:IftruethenthecollectiondatawillbekeptinmemoryonlyandArangoDBwillnotwriteorsyncthedatatodisk.This
optionismeaningfulfortheM M Filesstorageengineonly.
keyOptions(optional)additionaloptionsforkeygeneration.ThisisaJSONarraycontainingthefollowingattributes(note:someof
theattributesareoptional):
type:thetypeofthekeygeneratorusedforthecollection.
allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,
48
CollectionM ethods
thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeof
documentsisconsideredanerror.
increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
indexBuckets:numberofbucketsintowhichindexesusingahashtablearesplit.Thedefaultis16andthisnumberhastobeapower
of2andlessthanorequalto1024.ThisoptionismeaningfulfortheM M Filesstorageengineonly.Forverylargecollectionsone
shouldincreasethistoavoidlongpauseswhenthehashtablehastobeinitiallybuiltorresized,sincebucketsareresized
individuallyandcanbeinitiallybuiltinparallel.Forexample,64mightbeasensiblevalueforacollectionwith100000000
documents.Currently,onlytheedgeindexrespectsthisvalue,butotherindextypesmightfollowinfutureArangoDBversions.
Changes(seebelow)areappliedwhenthecollectionisloadedthenexttime.Inaclustersetup,theresultwillalsocontainthe
followingattributes:
numberOfShards:thenumberofshardsofthecollection.
shardKeys:containsthenamesofdocumentattributesthatareusedtodeterminethetargetshardfordocuments.
collection.properties(properties)Changesthecollectionproperties.propertiesmustbeaobjectwithoneormoreofthe
followingattribute(s):
waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheM M Filesstorageengineonly.
indexBuckets:Seeabove,changesareonlyappliedwhenthecollectionisloadedthenexttime.Thisoptionismeaningfulforthe
M M Filesstorageengineonly.Note:itisnotpossibletochangethejournalsizeafterthejournalordatafilehasbeencreated.
Changingthisparameterwillonlyeffectnewlycreatedjournals.Alsonotethatyoucannotlowerthejournalsizetolessthensizeof
thelargestdocumentalreadystoredinthecollection.Note:someothercollectionproperties,suchastype,isVolatile,orkeyOptions
cannotbechangedoncethecollectioniscreated.
Examples
Readallproperties
arangosh>db.example.properties();
showexecutionresults
Changeaproperty
arangosh>db.example.properties({waitForSync:true});
showexecutionresults
Figures
returnsthefiguresofacollection collection.figures()
Returnsanobjectcontainingstatisticsaboutthecollection.Note:Retrievingthefigureswillalwaysloadthecollectionintomemory.
alive.count:Thenumberofcurrentlyactivedocumentsinalldatafilesandjournalsofthecollection.Documentsthatarecontainedin
thewrite-aheadlogonlyarenotreportedinthisfigure.
alive.size:Thetotalsizeinbytesusedbyallactivedocumentsofthecollection.Documentsthatarecontainedinthewrite-aheadlog
onlyarenotreportedinthisfigure.
dead.count:Thenumberofdeaddocuments.Thisincludesdocumentversionsthathavebeendeletedorreplacedbyanewerversion.
Documentsdeletedorreplacedthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.
dead.size:Thetotalsizeinbytesusedbyalldeaddocuments.
dead.deletion:Thetotalnumberofdeletionmarkers.Deletionmarkersonlycontainedinthewrite-aheadlogarenotreportinginthis
figure.
datafiles.count:Thenumberofdatafiles.
datafiles.fileSize:Thetotalfilesizeofdatafiles(inbytes).
journals.count:Thenumberofjournalfiles.
journals.fileSize:Thetotalfilesizeofthejournalfiles(inbytes).
compactors.count:Thenumberofcompactorfiles.
compactors.fileSize:Thetotalfilesizeofthecompactorfiles(inbytes).
49
CollectionM ethods
shapefiles.count:Thenumberofshapefiles.Thisvalueisdeprecatedandkeptforcompatibilityreasonsonly.Thevaluewillalways
be0sinceArangoDB2.0andhigher.
shapefiles.fileSize:Thetotalfilesizeoftheshapefiles.Thisvalueisdeprecatedandkeptforcompatibilityreasonsonly.Thevalue
willalwaysbe0inArangoDB2.0andhigher.
shapes.count:Thetotalnumberofshapesusedinthecollection.Thisincludesshapesthatarenotinuseanymore.Shapesthatare
containedinthewrite-aheadlogonlyarenotreportedinthisfigure.
shapes.size:Thetotalsizeofallshapes(inbytes).Thisincludesshapesthatarenotinuseanymore.Shapesthatarecontainedin
thewrite-aheadlogonlyarenotreportedinthisfigure.
attributes.count:Thetotalnumberofattributesusedinthecollection.Note:thevalueincludesdataofattributesthatarenotinuse
anymore.Attributesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.
attributes.size:Thetotalsizeoftheattributedata(inbytes).Note:thevalueincludesdataofattributesthatarenotinuseanymore.
Attributesthatarecontainedinthewrite-aheadlogonlyarenotreportedinthisfigure.
indexes.count:Thetotalnumberofindexesdefinedforthecollection,includingthepre-definedindexes(e.g.primaryindex).
indexes.size:Thetotalmemoryallocatedforindexesinbytes.
lastTick:Thetickofthelastmarkerthatwasstoredinajournalofthecollection.Thismightbe0ifthecollectiondoesnotyethave
ajournal.
uncollectedLogfileEntries:Thenumberofmarkersinthewrite-aheadlogforthiscollectionthathavenotbeentransferredtojournals
ordatafiles.
documentReferences:ThenumberofreferencestodocumentsindatafilesthatJavaScriptcodecurrentlyholds.Thisinformationcan
beusedfordebuggingcompactionandunloadissues.
waitingFor:Anoptionalstringvaluethatcontainsinformationaboutwhichobjecttypeisattheheadofthecollection'scleanup
queue.Thisinformationcanbeusedfordebuggingcompactionandunloadissues.
compactionStatus.time:Thepointintimethecompactionforthecollectionwaslastexecuted.Thisinformationcanbeusedfor
debuggingcompactionissues.
compactionStatus.message:Theactionthatwasperformedwhenthecompactionwaslastrunforthecollection.Thisinformation
canbeusedfordebuggingcompactionissues.
Note:collectiondatathatarestoredinthewrite-aheadlogonlyarenotreportedintheresults.Whenthewrite-aheadlogiscollected,
documentsmightbeaddedtojournalsanddatafilesofthecollection,whichmaymodifythefiguresofthecollection.Alsonotethat
waitingForand compactionStatusmaybeemptywhencalledonacoordinatorinacluster.
Additionally,thefilesizesofcollectionandindexparameterJSONfilesarenotreported.Thesefilesshouldnormallyhaveasizeofafew
byteseach.PleasealsonotethatthefileSizevaluesarereportedinbytesandreflectthelogicalfilesizes.Somefilesystemsmayuse
optimisations(e.g.sparsefiles)sothattheactualphysicalfilesizeissomewhatdifferent.Directoriesandsub-directoriesmayalso
requirespaceinthefilesystem,butthisspaceisnotreportedinthefileSizeresults.
Thatmeansthatthefiguresreporteddonotreflecttheactualdiskusageofthecollectionwith100%accuracy.Theactualdiskusageofa
collectionisnormallyslightlyhigherthanthesumofthereportedfileSizevalues.StillthesumofthefileSizevaluescanstillbeusedasa
lowerboundapproximationofthediskusage.
Examples
arangosh>db.demo.figures()
showexecutionresults
Load
loadsacollection collection.load()
Loadsacollectionintomemory.
Note:clustercollectionsareloadedatalltimes.
Examples
arangosh>col=db.example;
[ArangoCollection16456,"example"(typedocument,statusloaded)]
arangosh>col.load();
50
CollectionM ethods
arangosh>col;
[ArangoCollection16456,"example"(typedocument,statusloaded)]
Revision
returnstherevisionidofacollection collection.revision()
Returnstherevisionidofthecollection
Therevisionidisupdatedwhenthedocumentdataismodified,eitherbyinserting,deleting,updatingorreplacingdocumentsinit.
Therevisionidofacollectioncanbeusedbyclientstocheckwhetherdatainacollectionhaschangedorifitisstillunmodifiedsincea
previousfetchoftherevisionid.
Therevisionidreturnedisastringvalue.Clientsshouldtreatthisvalueasanopaquestring,andonlyuseitforequality/non-equality
comparisons.
Path
returnsthephysicalpathofthecollection collection.path()
Thepathoperationreturnsastringwiththephysicalstoragepathforthecollectiondata.
Note:thismethodwillreturnnothingmeaningfulinacluster.Inasingle-serverArangoDB,thismethodwillonlyreturnmeaningfuldata
fortheM M Filesengine.
Checksum
calculatesachecksumforthedatainacollection collection.checksum(withRevisions,withData)
Thechecksumoperationcalculatesanaggregatehashvalueforalldocumentkeyscontainedincollectioncollection.
IftheoptionalargumentwithRevisionsissettotrue,thentherevisionidsofthedocumentsarealsoincludedinthehashcalculation.
IftheoptionalargumentwithDataissettotrue,thenalluser-defineddocumentattributesarealsochecksummed.Includingthedocument
datainchecksummingwillmakethecalculationslower,butismoreaccurate.
ThechecksumcalculationalgorithmchangedinArangoDB3.0,sochecksumsfrom3.0andearlierversionsforthesamedatawilldiffer.
Note:thismethodisnotavailableinacluster.
Unload
unloadsacollection collection.unload()
Startsunloadingacollectionfrommemory.Notethatunloadingisdeferreduntilallqueryhavefinished.
Note:clustercollectionscannotbeunloaded.
Examples
arangosh>col=db.example;
[ArangoCollection7423,"example"(typedocument,statusloaded)]
arangosh>col.unload();
arangosh>col;
[ArangoCollection7423,"example"(typedocument,statusunloaded)]
Rename
renamesacollection collection.rename(new-name)
Renamesacollectionusingthenew-name.Thenew-namemustnotalreadybeusedforadifferentcollection.new-namemustalsobea
validcollectionname.Formoreinformationonvalidcollectionnamespleaserefertothenamingconventions.
51
CollectionM ethods
Ifrenamingfailsforanyreason,anerroristhrown.Ifrenamingthecollectionsucceeds,thenthecollectionisalsorenamedinallgraph
definitionsinsidethe _graphscollectioninthecurrentdatabase.
Note:thismethodisnotavailableinacluster.
Examples
arangosh>c=db.example;
[ArangoCollection16569,"example"(typedocument,statusloaded)]
arangosh>c.rename("better-example");
arangosh>c;
[ArangoCollection16569,"better-example"(typedocument,statusloaded)]
Rotate
rotatesthecurrentjournalofacollection collection.rotate()
Rotatesthecurrentjournalofacollection.Thisoperationmakesthecurrentjournalofthecollectionaread-onlydatafilesoitmay
becomeacandidateforgarbagecollection.Ifthereiscurrentlynojournalavailableforthecollection,theoperationwillfailwithanerror.
Note:thismethodisspecificfortheM M Filesstorageengine,andthereitisnotavailableinacluster.
52
DatabaseM ethods
DatabaseMethods
Collection
returnsasinglecollectionornull db._collection(collection-name)
Returnsthecollectionwiththegivennameornullifnosuchcollectionexists.
db._collection(collection-identifier)
Returnsthecollectionwiththegivenidentifierornullifnosuchcollectionexists.Accessingcollectionsbyidentifierisdiscouragedfor
endusers.Endusersshouldaccesscollectionsusingthecollectionname.
Examples
Getacollectionbyname:
arangosh>db._collection("demo");
[ArangoCollection84,"demo"(typedocument,statusloaded)]
Getacollectionbyid:
arangosh>db._collection(123456);
[ArangoCollection123456,"demo"(typedocument,statusloaded)]
Unknowncollection:
arangosh>db._collection("unknown");
null
Create
createsanewdocumentoredgecollection db._create(collection-name)
Createsanewdocumentcollectionnamedcollection-name.Ifthecollectionnamealreadyexistsorifthenameformatisinvalid,anerroris
thrown.Formoreinformationonvalidcollectionnamespleaserefertothenamingconventions.
db._create(collection-name,properties)
propertiesmustbeanobjectwiththefollowingattributes:
waitForSync(optional,defaultfalse):Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize(optional,defaultisaconfigurationparameter:Themaximalsizeofajournalordatafile.Notethatthisalsolimitsthe
maximalsizeofasingleobject.M ustbeatleast1M B.
isSystem(optional,defaultisfalse):Iftrue,createasystemcollection.Inthiscasecollection-nameshouldstartwithanunderscore.
Endusersshouldnormallycreatenon-systemcollectionsonly.APIimplementorsmayberequiredtocreatesystemcollectionsin
veryspecialoccasions,butnormallyaregularcollectionwilldo.
isVolatile(optional,defaultisfalse):Iftruethenthecollectiondataiskeptin-memoryonlyandnotmadepersistent.Unloadingthe
collectionwillcausethecollectiondatatobediscarded.Stoppingorre-startingtheserverwillalsocausefulllossofdatainthe
collection.Thecollectionitselfwillremainhowever(onlythedataisvolatile).Settingthisoptionwillmaketheresultingcollection
beslightlyfasterthanregularcollectionsbecauseArangoDBdoesnotenforceanysynchronizationtodiskanddoesnotcalculate
anyCRCchecksumsfordatafiles(astherearenodatafiles).ThisoptionismeaningfulfortheM M Filesstorageengineonly.
keyOptions(optional):additionaloptionsforkeygeneration.Ifspecified,thenkeyOptionsshouldbeaJSONobjectcontainingthe
followingattributes(note:someofthemareoptional):
type:specifiesthetypeofthekeygenerator.Thecurrentlyavailablegeneratorsaretraditionalandautoincrement.(note:
autoincrementiscurrentlyonlysupportedfornon-shardedcollections)
53
DatabaseM ethods
allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,
thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeof
documentsisconsideredanerror.
increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
numberOfShards(optional,defaultis1):inacluster,thisvaluedeterminesthenumberofshardstocreateforthecollection.Ina
singleserversetup,thisoptionismeaningless.
shardKeys(optional,defaultis ["_key"]):inacluster,thisattributedetermineswhichdocumentattributesareusedtodetermine
thetargetshardfordocuments.Documentsaresenttoshardsbasedonthevaluestheyhaveintheirshardkeyattributes.Thevalues
ofallshardkeyattributesinadocumentarehashed,andthehashvalueisusedtodeterminethetargetshard.Notethatvaluesof
shardkeyattributescannotbechangedonceset.Thisoptionismeaninglessinasingleserversetup.
Whenchoosingtheshardkeys,onemustbeawareofthefollowingrulesandlimitations:Inashardedcollectionwithmorethanone
sharditisnotpossibletosetupauniqueconstraintonanattributethatisnottheoneandonlyshardkeygiveninshardKeys.This
isbecauseenforcingauniqueconstraintwouldotherwisemakeaglobalindexnecessaryorneedextensivecommunicationforevery
singlewriteoperation.Furthermore,if_keyisnottheoneandonlyshardkey,thenitisnotpossibletosetthe_keyattributewhen
insertingadocument,providedthecollectionhasmorethanoneshard.Again,thisisbecausethedatabasehastoenforcetheunique
constraintonthe_keyattributeandthiscanonlybedoneefficientlyifthisistheonlyshardkeybydelegatingtotheindividual
shards.
replicationFactor(optional,defaultis1):inacluster,thisattributedetermineshowmanycopiesofeachshardarekeptondifferent
DBServers.Thevalue1meansthatonlyonecopy(nosynchronousreplication)iskept.Avalueofkmeansthatk-1replicasare
kept.AnytwocopiesresideondifferentDBServers.Replicationbetweenthemissynchronous,thatis,everywriteoperationtothe
"leader"copywillbereplicatedtoall"follower"replicas,beforethewriteoperationisreportedsuccessful.
Ifaserverfails,thisisdetectedautomaticallyandoneoftheserversholdingcopiestakeover,usuallywithoutanerrorbeing
reported.
distributeShardsLikedistributetheshardsofthiscollectioncloningthesharddistributionofanother.
WhenusingtheEnterpriseversionofArangoDBthereplicationFactormaybesetto"satellite"makingthecollectionlocallyjoinable
oneverydatabaseserver.ThisreducesthenumberofnetworkhopsdramaticallywhenusingjoinsinAQLatthecostsofreduced
writeperformanceonthesecollections.
db._create(collection-name,properties,type)
Specifiestheoptionaltypeofthecollection,itcaneitherbedocumentoredge.Ondefaultitisdocument.Insteadofgivingatypeyoucan
alsousedb._createEdgeCollectionordb._createDocumentCollection.
Examples
Withdefaults:
arangosh>c=db._create("users");
arangosh>c.properties();
showexecutionresults
Withproperties:
arangosh>c=db._create("users",{waitForSync:true,
........>journalSize:1024*1204});
arangosh>c.properties();
showexecutionresults
Withakeygenerator:
arangosh>db._create("users",
........>{keyOptions:{type:"autoincrement",offset:10,increment:5}});
arangosh>db.users.save({name:"user1"});
54
DatabaseM ethods
arangosh>db.users.save({name:"user2"});
arangosh>db.users.save({name:"user3"});
showexecutionresults
Withaspecialkeyoption:
arangosh>db._create("users",{keyOptions:{allowUserKeys:false}});
arangosh>db.users.save({name:"user1"});
arangosh>db.users.save({name:"user2",_key:"myuser"});
arangosh>db.users.save({name:"user3"});
showexecutionresults
createsanewedgecollection db._createEdgeCollection(collection-name)
Createsanewedgecollectionnamedcollection-name.Ifthecollectionnamealreadyexistsanerroristhrown.Thedefaultvaluefor
waitForSyncisfalse.
db._createEdgeCollection(collection-name,properties)
propertiesmustbeanobjectwiththefollowingattributes:
waitForSync(optional,defaultfalse):Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize(optional,defaultis"configurationparameter"):Themaximalsizeofajournalordatafile.Notethatthisalsolimitsthe
maximalsizeofasingleobjectandmustbeatleast1M B.
createsanewdocumentcollection db._createDocumentCollection(collection-name)
Createsanewdocumentcollectionnamedcollection-name.Ifthedocumentnamealreadyexistsanderroristhrown.
AllCollections
returnsallcollections db._collections()
Returnsallcollectionsofthegivendatabase.
Examples
arangosh>db._collections();
showexecutionresults
CollectionName
selectsacollectionfromthevocbase db.collection-name
Returnsthecollectionwiththegivencollection-name.Ifnosuchcollectionexists,createacollectionnamedcollection-namewiththe
defaultproperties.
Examples
arangosh>db.example;
[ArangoCollection16155,"example"(typedocument,statusloaded)]
Drop
dropsacollection db._drop(collection)
Dropsacollectionandallitsindexesanddata.
db._drop(collection-identifier)
Dropsacollectionidentifiedbycollection-identifierwithallitsindexesanddata.Noerroristhrownifthereisnosuchcollection.
55
DatabaseM ethods
db._drop(collection-name)
Dropsacollectionnamedcollection-nameandallitsindexes.Noerroristhrownifthereisnosuchcollection.
db._drop(collection-name,options)
Inordertodropasystemcollection,onemustspecifyanoptionsobjectwithattributeisSystemsettotrue.Otherwiseitisnotpossible
todropsystemcollections.
Note:clustercollection,whichareprototypesforcollectionswithdistributeShardsLikeparameter,cannotbedropped.
Examples
Dropsacollection:
arangosh>col=db.example;
[ArangoCollection16247,"example"(typedocument,statusloaded)]
arangosh>db._drop(col);
arangosh>col;
[ArangoCollection16247,"example"(typedocument,statusloaded)]
Dropsacollectionidentifiedbyname:
arangosh>col=db.example;
[ArangoCollection16257,"example"(typedocument,statusloaded)]
arangosh>db._drop("example");
arangosh>col;
[ArangoCollection16257,"example"(typedocument,statusdeleted)]
Dropsasystemcollection
arangosh>col=db._example;
[ArangoCollection16267,"_example"(typedocument,statusloaded)]
arangosh>db._drop("_example",{isSystem:true});
arangosh>col;
[ArangoCollection16267,"_example"(typedocument,statusdeleted)]
Truncate
truncatesacollection db._truncate(collection)
Truncatesacollection,removingalldocumentsbutkeepingallitsindexes.
db._truncate(collection-identifier)
Truncatesacollectionidentifiedbycollection-identified.Noerroristhrownifthereisnosuchcollection.
db._truncate(collection-name)
Truncatesacollectionnamedcollection-name.Noerroristhrownifthereisnosuchcollection.
Examples
Truncatesacollection:
arangosh>col=db.example;
arangosh>col.save({"Hello":"World"});
arangosh>col.count();
arangosh>db._truncate(col);
arangosh>col.count();
showexecutionresults
56
DatabaseM ethods
Truncatesacollectionidentifiedbyname:
arangosh>col=db.example;
arangosh>col.save({"Hello":"World"});
arangosh>col.count();
arangosh>db._truncate("example");
arangosh>col.count();
showexecutionresults
57
Documents
Documents
ThisisanintroductiontoArangoDB'sinterfaceforworkingwithdocumentsfromtheJavaScriptshellarangoshorinJavaScriptcodein
theserver.ForotherlanguagesseethecorrespondinglanguageAPI.
BasicsandTerminology:sectiononthebasicapproach
CollectionM ethods:detailedAPIdescriptionforcollectionobjects
DatabaseM ethods:detailedAPIdescriptionfordatabaseobjects
58
BasicsandTerminology
BasicsandTerminology
DocumentsinArangoDBareJSONobjects.Theseobjectscanbenested(toanydepth)andmaycontainlists.Eachdocumenthasa
uniqueprimarykeywhichidentifiesitwithinitscollection.Furthermore,eachdocumentisuniquelyidentifiedbyitsdocumenthandle
acrossallcollectionsinthesamedatabase.Differentrevisionsofthesamedocument(identifiedbyitshandle)canbedistinguishedby
theirdocumentrevision.Anytransactiononlyeverseesasinglerevisionofadocument.Forexample:
{
"_id":"myusers/3456789",
"_key":"3456789",
"_rev":"14253647",
"firstName":"John",
"lastName":"Doe",
"address":{
"street":"RoadToNowhere1",
"city":"Gotham"
},
"hobbies":[
{name:"swimming",howFavorite:10},
{name:"biking",howFavorite:6},
{name:"programming",howFavorite:4}
]
}
Alldocumentscontainspecialattributes:thedocumenthandleisstoredasastringin _id,thedocument'sprimarykeyin _keyand
thedocumentrevisionin _rev.Thevalueofthe _keyattributecanbespecifiedbytheuserwhencreatingadocument. _idand
_keyvaluesareimmutableoncethedocumenthasbeencreated.The _revvalueismaintainedbyArangoDBautomatically.
DocumentHandle
Adocumenthandleuniquelyidentifiesadocumentinthedatabase.Itisastringandconsistsofthecollection'snameandthedocument
key( _keyattribute)separatedby /.
DocumentKey
Adocumentkeyuniquelyidentifiesadocumentinthecollectionitisstoredin.Itcanandshouldbeusedbyclientswhenspecific
documentsarequeried.Thedocumentkeyisstoredinthe _keyattributeofeachdocument.Thekeyvaluesareautomaticallyindexed
byArangoDBinacollection'sprimaryindex.Thuslookingupadocumentbyitskeyisafastoperation.The_keyvalueofadocumentis
immutableoncethedocumenthasbeencreated.Bydefault,ArangoDBwillauto-generateadocumentkeyifno_keyattributeis
specified,andusetheuser-specified_keyotherwise.Thegenerated_keyisguaranteedtobeuniqueinthecollectionitwasgeneratedfor.
Thisalsoappliestoshardedcollectionsinacluster.Itcan'tbeguaranteedthatthe_keyisuniquewithinadatabaseoracrossawhole
nodeorinstancehowever.
Thisbehaviorcanbechangedonaper-collectionlevelbycreatingcollectionswiththe keyOptionsattribute.
Using keyOptionsitispossibletodisallowuser-specifiedkeyscompletely,ortoforceaspecificregimeforauto-generatingthe _key
values.
DocumentRevision
AsArangoDBsupportsM VCC(M ultipleVersionConcurrencyControl),documentscanexistinmorethanonerevision.Thedocument
revisionistheM VCCtokenusedtospecifyaparticularrevisionofadocument(identifiedbyits _id).Itisastringvaluethat
contained(uptoArangoDB3.0)anintegernumberandisuniquewithinthelistofdocumentrevisionsforasingledocument.In
ArangoDB>=3.1the_revstringsareinfacttimestamps.TheyusethelocalclockoftheDBserverthatactuallywritesthedocument
andhavemillisecondaccuracy.Actually,a"HybridLogicalClock"isused(forthisconceptseethispaper).
Withinonesharditisguaranteedthattwodifferentdocumentrevisionshaveadifferent_revstring,eveniftheyarewritteninthesame
millisecond,andthatthesestampsareascending.
59
BasicsandTerminology
Notehoweverthatdifferentserversinyourclustermighthaveaclockskew,andthereforebetweendifferentshardsorevenbetween
differentcollectionsthetimestampsarenotguaranteedtobecomparable.
TheHybridLogicalClockfeaturedoesonethingtoaddressthisissue:WheneveramessageissentfromsomeserverAinyourclusterto
anotheroneB,itisensuredthatanytimestamptakenonBafterthemessagehasarrivedisgreaterthananytimestamptakenonAbefore
themessagewassent.Thisensuresthatifthereissome"causality"betweeneventsondifferentservers,timestampsincreasefromcause
toeffect.Adirectconsequenceofthisisthatsometimesaserverhastotaketimestampsthatseemtocomefromthefutureofitsown
clock.Itwillhoweverstillproduceeverincreasingtimestamps.Iftheclockskewissmall,thenyourtimestampswillrelativelyaccurately
describethetimewhenthedocumentrevisionwasactuallywritten.
ArangoDBuses64bitunsignedintegervaluestomaintaindocumentrevisionsinternally.Atthisstageweintentionallydonotdocument
theexactformatoftherevisionvalues.Whenreturningdocumentrevisionstoclients,ArangoDBwillputthemintoastringtoensurethe
revisionisnotclippedbyclientsthatdonotsupportbigintegers.ClientsshouldtreattherevisionreturnedbyArangoDBasanopaque
stringwhentheystoreoruseitlocally.ThiswillallowArangoDBtochangetheformatofrevisionslaterifthisshouldberequired(ashas
happenedwith3.1withtheHybridLogicalClock).Clientscanuserevisionstoperformsimpleequality/non-equalitycomparisons(e.g.
tocheckwhetheradocumenthaschangedornot),buttheyshouldnotuserevisionidstoperformgreater/lessthancomparisonswith
themtocheckifadocumentrevisionisolderthanoneanother,evenifthismightworkforsomecases.
Documentrevisionscanbeusedtoconditionallyquery,update,replaceordeletedocumentsinthedatabase.Inordertofindaparticular
revisionofadocument,youneedthedocumenthandleorkey,andthedocumentrevision.
MultipleDocumentsinasingleCommand
BeginningwithArangoDB3.0thebasicdocumentAPIhasbeenextendedtohandlenotonlysingledocumentsbutmultipledocumentsin
asinglecommand.Thisiscrucialforperformance,inparticularintheclustersituation,inwhichasinglerequestcaninvolvemultiple
networkhopswithinthecluster.Anotheradvantageisthatitreducestheoverheadofindividualnetworkroundtripsbetweentheclient
andtheserver.ThegeneralideatoperformmultipledocumentoperationsinasinglecommandistouseJSONarraysofobjectsinthe
placeofasingledocument.Asaconsequence,documentkeys,handlesandrevisionsforpreconditionshavetobesuppliedembeddedin
theindividualdocumentsgiven.M ultipledocumentoperationsarerestrictedtoasingledocumentoredgecollection.SeetheAPI
descriptionsforcollectionobjectsfordetails.NotethattheAPIfordatabaseobjectsdonotoffertheseoperations.
60
CollectionM ethods
CollectionMethods
All
collection.all()
Fetchesalldocumentsfromacollectionandreturnsacursor.YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbe
limitedusingtheskipandlimitoperator.
Examples
UsetoArraytogetalldocumentsatonce:
arangosh>db.five.save({name:"one"});
arangosh>db.five.save({name:"two"});
arangosh>db.five.save({name:"three"});
arangosh>db.five.save({name:"four"});
arangosh>db.five.save({name:"five"});
arangosh>db.five.all().toArray();
showexecutionresults
Uselimittorestrictthedocuments:
arangosh>db.five.save({name:"one"});
arangosh>db.five.save({name:"two"});
arangosh>db.five.save({name:"three"});
arangosh>db.five.save({name:"four"});
arangosh>db.five.save({name:"five"});
arangosh>db.five.all().limit(2).toArray();
showexecutionresults
Querybyexample
collection.byExample(example)
Fetchesalldocumentsfromacollectionthatmatchthespecifiedexampleandreturnsacursor.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.Ifyouuse
{"a":{"c":1}}
asexample,thenyouwillfindalldocuments,suchthattheattributeacontainsadocumentoftheform{c:1}.Forexamplethe
document
{"a":{"c":1},"b":1}
willmatch,butthedocument
{"a":{"c":1,"b":1}}
willnot.
However,ifyouuse
{"a.c":1}
61
CollectionM ethods
thenyouwillfindalldocuments,whichcontainasub-documentinathathasanattributecofvalue1.Boththefollowingdocuments
{"a":{"c":1},"b":1}
and
{"a":{"c":1,"b":1}}
willmatch.
collection.byExample(path1,value1,...)
Asalternativeyoucansupplyanarrayofpathsandvalues.
Examples
UsetoArraytogetalldocumentsatonce:
arangosh>db.users.save({name:"Gerhard"});
arangosh>db.users.save({name:"Helmut"});
arangosh>db.users.save({name:"Angela"});
arangosh>db.users.all().toArray();
arangosh>db.users.byExample({"_id":"users/20"}).toArray();
arangosh>db.users.byExample({"name":"Gerhard"}).toArray();
arangosh>db.users.byExample({"name":"Helmut","_id":"users/15"}).toArray();
showexecutionresults
Usenexttoloopoveralldocuments:
arangosh>db.users.save({name:"Gerhard"});
arangosh>db.users.save({name:"Helmut"});
arangosh>db.users.save({name:"Angela"});
arangosh>vara=db.users.byExample({"name":"Angela"});
arangosh>while(a.hasNext())print(a.next());
showexecutionresults
FirstExample
collection.firstExample(example)
Returnssomedocumentofacollectionthatmatchesthespecifiedexample.Ifnosuchdocumentexists,nullwillbereturned.Theexample
hastobespecifiedaspathsandvalues.SeebyExamplefordetails.
collection.firstExample(path1,value1,...)
Asalternativeyoucansupplyanarrayofpathsandvalues.
Examples
arangosh>db.users.firstExample("name","Angela");
showexecutionresults
Range
collection.range(attribute,left,right)
62
CollectionM ethods
Returnsalldocumentsfromacollectionsuchthattheattributeisgreaterorequalthanleftandstrictlylessthanright.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.
Note:therangesimplequeryfunctionisdeprecatedasofArangoDB2.6.Thefunctionmayberemovedinfutureversionsof
ArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionwithinaspecificrangeistouseanAQLqueryasfollows:
FORdocIN@@collection
FILTERdoc.value>=@left&&doc.value<@right
LIMIT@skip,@limit
RETURNdoc
Examples
UsetoArraytogetalldocumentsatonce:
arangosh>db.old.ensureIndex({type:"skiplist",fields:["age"]});
arangosh>db.old.save({age:15});
arangosh>db.old.save({age:25});
arangosh>db.old.save({age:30});
arangosh>db.old.range("age",10,30).toArray();
showexecutionresults
Closedrange
collection.closedRange(attribute,left,right)
Returnsalldocumentsofacollectionsuchthattheattributeisgreaterorequalthanleftandlessorequalthanright.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.
Note:theclosedRangesimplequeryfunctionisdeprecatedasofArangoDB2.6.Thefunctionmayberemovedinfutureversionsof
ArangoDB.ThepreferredwayforretrievingdocumentsfromacollectionwithinaspecificrangeistouseanAQLqueryasfollows:
FORdocIN@@collection
FILTERdoc.value>=@left&&doc.value<=@right
LIMIT@skip,@limit
RETURNdoc
Examples
UsetoArraytogetalldocumentsatonce:
arangosh>db.old.ensureIndex({type:"skiplist",fields:["age"]});
arangosh>db.old.save({age:15});
arangosh>db.old.save({age:25});
arangosh>db.old.save({age:30});
arangosh>db.old.closedRange("age",10,30).toArray();
showexecutionresults
Any
collection.any()
Returnsarandomdocumentfromthecollectionornullifnoneexists.
Note:thismethodisexpensivewhenusingtheRocksDBstorageengine.
63
CollectionM ethods
Count
collection.count()
Returnsthenumberoflivingdocumentsinthecollection.
Examples
arangosh>db.users.count();
0
toArray
collection.toArray()
Convertsthecollectionintoanarrayofdocuments.Neverusethiscallinaproductionenvironmentasitwillbasicallycreateacopyof
yourcollectioninRAM whichwilluseresourcesdependingonthenumberandsizeofthedocumentsinyourcollecion.
Document
collection.document(object)
Thedocumentmethodfindsadocumentgivenanobjectobjectcontainingthe_idor_keyattribute.Themethodreturnsthedocumentifit
canbefound.Ifbothattributesaregiven,the_idtakesprecedence,itisanerror,ifthecollectionpartofthe_iddoesnotmatchthe
collection.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.Anerrorisalsothrownifnodocument
existswiththegiven_idor_keyvalue.
Pleasenotethatifthemethodisexecutedonthearangodserver(e.g.frominsideaFoxxapplication),animmutabledocumentobjectwill
bereturnedforperformancereasons.Itisnotpossibletochangeattributesofthisimmutableobject.Toupdateorpatchthereturned
document,itneedstobecloned/copiedintoaregularJavaScriptobjectfirst.Thisisnotnecessaryifthedocumentmethodiscalledfrom
outofarangoshorfromanyotherclient.
collection.document(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.Norevisioncanbespecifiedinthiscase.
collection.document(document-key)
Asbefore.Insteadofobjectadocument-keycanbepassedasfirstargument.
collection.document(array)
Thisvariantallowstoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifdocumentwouldhavebeen
calledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,no
exceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.
Examples
Returnsthedocumentforadocument-handle:
arangosh>db.example.document("example/2873916");
showexecutionresults
Returnsthedocumentforadocument-key:
arangosh>db.example.document("2873916");
showexecutionresults
Returnsthedocumentforanobject:
arangosh>db.example.document({_id:"example/2873916"});
64
CollectionM ethods
showexecutionresults
Returnsthedocumentforanarrayoftwokeys:
arangosh>db.example.document(["2873916","2873917"]);
showexecutionresults
Anerrorisraisedifthedocumentisunknown:
arangosh>db.example.document("example/4472917");
[ArangoError1202:documentnotfound]
Anerrorisraisedifthehandleisinvalid:
arangosh>db.example.document("");
[ArangoError1205:illegaldocumenthandle]
Changesin3.0from2.8:
documentcannowquerymultipledocumentswithonecall.
Exists
checkswhetheradocumentexists collection.exists(object)
Theexistsmethoddetermineswhetheradocumentexistsgivenanobject objectcontainingthe_idor_keyattribute.Ifbothattributes
aregiven,the_idtakesprecedence,itisanerror,ifthecollectionpartofthe_iddoesnotmatchthecollection.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.
Insteadofreturningthefounddocumentoranerror,thismethodwillonlyreturnanobjectwiththeattributes_id,_keyand_rev,orfalse
ifnodocumentwiththegiven_idor_keyexists.Itcanthusbeusedforeasyexistencechecks.
Thismethodwillthrowanerrorifusedimproperly,e.g.whencalledwithanon-documenthandle,anon-document,orwhenacrosscollectionrequestisperformed.
collection.exists(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.
collection.exists(document-key)
Asbefore.Insteadofobjectadocument-keycanbepassedasfirstargument.
collection.exists(array)
Thisvariantallowstoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifexistswouldhavebeencalledon
allmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,theoperation
stopsimmediatelyreturningonlyanerrorobject.
Changesin3.0from2.8:
Inthecaseofarevisionmismatchexistsnowthrowsanerrorinsteadofsimplyreturningfalse.Thisistomakeitpossibletotellthe
differencebetweenarevisionmismatchandanon-existingdocument.
existscannowquerymultipledocumentswithonecall.
LookupByKeys
collection.documents(keys)
65
CollectionM ethods
Looksupthedocumentsinthespecifiedcollectionusingthearrayofkeysprovided.Alldocumentsforwhichamatchingkeywas
specifiedinthekeysarrayandthatexistinthecollectionwillbereturned.Keysforwhichnodocumentcanbefoundintheunderlying
collectionareignored,andnoexceptionwillbethrownforthem.
Thismethodisdeprecatedinfavourofthearrayvariantofdocument.
Examples
arangosh>keys=[];
arangosh>for(vari=0;i<10;++i){
........>db.example.insert({_key:"test"+i,value:i});
........>keys.push("test"+i);
........>}
arangosh>db.example.documents(keys);
showexecutionresults
Insert
collection.insert(data)
Createsanewdocumentinthecollectionfromthegivendata.Thedatamustbeanobject.Theattributes_idand_revareignoredandare
automaticallygenerated.Auniquevaluefortheattribute_keywillbeautomaticallygeneratedifnotspecified.Ifspecified,theremustnot
beadocumentwiththegiven_keyinthecollection.
Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleofthenewly
createddocument,theattribute_keythedocumentkeyandtheattribute_revcontainsthedocumentrevision.
collection.insert(data,options)
Createsanewdocumentinthecollectionfromthegivendataasabove.Theoptionaloptionsparametermustbeanobjectandcanbeused
tospecifythefollowingoptions:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagis
beendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecific
operations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthe
collection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collectionsthathaveadefaultwaitForSyncvalueoftrue.
silent:Ifthisflagissettotrue,themethoddoesnotreturnanyoutput.
returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.
Note:sinceArangoDB2.2,insertisanaliasforsave.
collection.insert(array)
collection.insert(array,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofarguments.Thebehaviorisexactlyasifinsertwouldhavebeen
calledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,no
exceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.
Changesin3.0from2.8:
TheoptionssilentandreturnNewarenew.Themethodcannowinsertmultipledocumentswithonecall.
Examples
arangosh>db.example.insert({Hello:"World"});
arangosh>db.example.insert({Hello:"World"},{waitForSync:true});
showexecutionresults
arangosh>db.example.insert([{Hello:"World"},{Hello:"there"}])
66
CollectionM ethods
arangosh>db.example.insert([{Hello:"World"},{}],{waitForSync:true});
showexecutionresults
Replace
collection.replace(selector,data)
Replacesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbea
documentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenreplacedwiththedatagivenassecondargument.Any
attribute_id,_keyor_revindataisignored.
Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleofthe
updateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainsthe
revisionoftheold(nowreplaced)document.
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,
thereisaconflict,andanerroristhrown.
collection.replace(selector,data,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagis
beendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecific
operations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthe
collection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collectionsthathaveadefaultwaitForSyncvalueoftrue.
overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.
returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.
returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.
silent:Ifthisflagissettotrue,nooutputisreturned.
collection.replace(document-handle,data)
collection.replace(document-handle,data,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.
collection.replace(document-key,data)
collection.replace(document-key,data,options)
Asbefore.Insteadofselectoradocument-keycanbepassedasfirstargument.Norevisionpreconditionistested.
collection.replace(selectorarray,dataarray)
collection.replace(selectorarray,dataarray,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofselector/datapairs.Thetwoarraysgivenasselectorarrayand
dataarraymusthavethesamelength.Thebehaviorisexactlyasifreplacewouldhavebeencalledonallrespectivemembersofthetwo
arraysandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocument
anerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.
Examples
Createandupdateadocument:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db.example.replace(a1,{a:2});
arangosh>a3=db.example.replace(a1,{a:3});
showexecutionresults
Useadocumenthandle:
arangosh>a1=db.example.insert({a:1});
67
CollectionM ethods
arangosh>a2=db.example.replace("example/3903044",{a:2});
showexecutionresults
Changesin3.0from2.8:
Theoptionssilent,returnNewandreturnOldarenew.Themethodcannowreplacemultipledocumentswithonecall.
Update
collection.update(selector,data)
Updatesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbea
documentwiththat_idor_keyinthecurrentcollection.Thisdocumentisthenpatchedwiththedatagivenassecondargument.Any
attribute_id,_keyor_revindataisignored.
Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleofthe
updateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainsthe
revisionoftheold(nowupdated)document.
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,
thereisaconflict,andanerroristhrown.
collection.update(selector,data,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagis
beendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecific
operations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthe
collection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collectionsthathaveadefaultwaitForSyncvalueoftrue.
overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.
returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.
returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.
silent:Ifthisflagissettotrue,nooutputisreturned.
keepNull:TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesare
storedinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnull
valueswillberemovedfromthetargetdocument.
mergeObjects:Controlswhetherobjects(notarrays)willbemergedifpresentinboththeexistingandthepatchdocument.Ifsetto
false,thevalueinthepatchdocumentwilloverwritetheexistingdocument'svalue.Ifsettotrue,objectswillbemerged.Thedefault
istrue.
collection.update(document-handle,data)
collection.update(document-handle,data,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.
collection.update(document-key,data)
collection.update(document-key,data,options)
Asbefore.Insteadofselectoradocument-keycanbepassedasfirstargument.Norevisionpreconditionistested.
collection.update(selectorarray,dataarray)
collection.update(selectorarray,dataarray,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofselector/datapairs.Thetwoarraysgivenasselectorarrayand
dataarraymusthavethesamelength.Thebehaviorisexactlyasifupdatewouldhavebeencalledonallrespectivemembersofthetwo
arraysandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,noexceptionisrisen!Insteadofadocument
anerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.
Examples
Createandupdateadocument:
68
CollectionM ethods
arangosh>a1=db.example.insert({"a":1});
arangosh>a2=db.example.update(a1,{"b":2,"c":3});
arangosh>a3=db.example.update(a1,{"d":4});
arangosh>a4=db.example.update(a2,{"e":5,"f":6});
arangosh>db.example.document(a4);
arangosh>a5=db.example.update(a4,{"a":1,c:9,e:42});
arangosh>db.example.document(a5);
showexecutionresults
Useadocumenthandle:
arangosh>a1=db.example.insert({"a":1});
arangosh>a2=db.example.update("example/18612115",{"x":1,"y":2});
showexecutionresults
UsethekeepNullparametertoremoveattributeswithnullvalues:
arangosh>db.example.insert({"a":1});
arangosh>db.example.update("example/19988371",
........>{"b":null,"c":null,"d":3});
arangosh>db.example.document("example/19988371");
arangosh>db.example.update("example/19988371",{"a":null},false,false);
arangosh>db.example.document("example/19988371");
arangosh>db.example.update("example/19988371",
........>{"b":null,"c":null,"d":null},false,false);
arangosh>db.example.document("example/19988371");
showexecutionresults
Patchingarrayvalues:
arangosh>db.example.insert({"a":{"one":1,"two":2,"three":3},
........>"b":{}});
arangosh>db.example.update("example/20774803",{"a":{"four":4},
........>"b":{"b1":1}});
arangosh>db.example.document("example/20774803");
arangosh>db.example.update("example/20774803",{"a":{"one":null},
........>"b":null},
........>false,false);
arangosh>db.example.document("example/20774803");
showexecutionresults
Changesin3.0from2.8:
Theoptionssilent,returnNewandreturnOldarenew.Themethodcannowupdatemultipledocumentswithonecall.
Remove
collection.remove(selector)
Removesadocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idor_keyattribute.Theremustbeadocument
withthat_idor_keyinthecurrentcollection.Thisdocumentisthenremoved.
Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleoftheremoved
document,theattribute_revcontainsthedocumentrevisionoftheremoveddocument.
69
CollectionM ethods
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,
thereisaconflict,andanerroristhrown.
collection.remove(selector,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagis
beendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecific
operations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthe
collection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collectionsthathaveadefaultwaitForSyncvalueoftrue.
overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.
returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.
silent:Ifthisflagissettotrue,nooutputisreturned.
collection.remove(document-handle)
collection.remove(document-handle,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisioncheckisperformed.
collection.remove(document-key)
collection.remove(document-handle,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisioncheckisperformed.
collection.remove(selectorarray)
collection.remove(selectorarray,options)
Thesetwovariantsallowtoperformtheoperationonawholearrayofselectors.Thebehaviorisexactlyasifremovewouldhavebeen
calledonallmembersofthearrayseparatelyandallresultsarereturnedinanarray.Ifanerroroccurswithanyofthedocuments,no
exceptionisrisen!Insteadofadocumentanerrorobjectisreturnedintheresultarray.Theoptionsbehaveexactlyasbefore.
Examples
Removeadocument:
arangosh>a1=db.example.insert({a:1});
arangosh>db.example.document(a1);
arangosh>db.example.remove(a1);
arangosh>db.example.document(a1);
showexecutionresults
Removeadocumentwithaconflict:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db.example.replace(a1,{a:2});
arangosh>db.example.remove(a1);
arangosh>db.example.remove(a1,true);
arangosh>db.example.document(a1);
showexecutionresults
Changesin3.0from2.8:
Themethodnowreturnsnotonlytruebutinformationabouttheremoveddocument(s).TheoptionssilentandreturnOldarenew.The
methodcannowremovemultipledocumentswithonecall.
RemoveByKeys
collection.removeByKeys(keys)
70
CollectionM ethods
Looksupthedocumentsinthespecifiedcollectionusingthearrayofkeysprovided,andremovesalldocumentsfromthecollection
whosekeysarecontainedinthekeysarray.Keysforwhichnodocumentcanbefoundintheunderlyingcollectionareignored,andno
exceptionwillbethrownforthem.
Themethodwillreturnanobjectcontainingthenumberofremoveddocumentsintheremovedsub-attribute,andthenumberofnotremoved/ignoreddocumentsintheignoredsub-attribute.
Thismethodisdeprecatedinfavourofthearrayvariantofremove.
Examples
arangosh>keys=[];
arangosh>for(vari=0;i<10;++i){
........>db.example.insert({_key:"test"+i,value:i});
........>keys.push("test"+i);
........>}
arangosh>db.example.removeByKeys(keys);
showexecutionresults
RemoveByExample
collection.removeByExample(example)
Removesalldocumentsmatchinganexample.
collection.removeByExample(document,waitForSync)
TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentdeletionoperationtodiskevenincasethat
thewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforcesynchronizationof
justspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,
thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collectionsthathaveadefaultwaitForSyncvalueoftrue.
collection.removeByExample(document,waitForSync,limit)
Theoptionallimitparametercanbeusedtorestrictthenumberofremovalstothespecifiedvalue.Iflimitisspecifiedbutlessthanthe
numberofdocumentsinthecollection,itisundefinedwhichdocumentsareremoved.
Examples
arangosh>db.example.removeByExample({Hello:"world"});
1
ReplaceByExample
collection.replaceByExample(example,newValue)
Replacesalldocumentsmatchinganexamplewithanewdocumentbody.Theentiredocumentbodyofeachdocumentmatchingthe
examplewillbereplacedwithnewValue.Thedocumentmeta-attributes_id,_keyand_revwillnotbereplaced.
collection.replaceByExample(document,newValue,waitForSync)
TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentreplacementoperationtodiskevenincase
thatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforce
synchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnot
specifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedto
disablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.
collection.replaceByExample(document,newValue,waitForSync,limit)
Theoptionallimitparametercanbeusedtorestrictthenumberofreplacementstothespecifiedvalue.Iflimitisspecifiedbutlessthan
thenumberofdocumentsinthecollection,itisundefinedwhichdocumentsarereplaced.
71
CollectionM ethods
Examples
arangosh>db.example.save({Hello:"world"});
arangosh>db.example.replaceByExample({Hello:"world"},{Hello:"mars"},false,5);
showexecutionresults
UpdateByExample
collection.updateByExample(example,newValue)
Partiallyupdatesalldocumentsmatchinganexamplewithanewdocumentbody.Specificattributesinthedocumentbodyofeach
documentmatchingtheexamplewillbeupdatedwiththevaluesfromnewValue.Thedocumentmeta-attributes_id,_keyand_revcannot
beupdated.
Partialupdatecouldalsobeusedtoappendnewfields,iftherewerenooldfieldwithsamename.
collection.updateByExample(document,newValue,keepNull,waitForSync)
TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesarestoredinthe
database.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnullvalueswillbe
removedfromthetargetdocument.
TheoptionalwaitForSyncparametercanbeusedtoforcesynchronizationofthedocumentreplacementoperationtodiskevenincase
thatthewaitForSyncflaghadbeendisabledfortheentirecollection.Thus,thewaitForSyncparametercanbeusedtoforce
synchronizationofjustspecificoperations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnot
specifiedorsettofalse,thenthecollection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedto
disablesynchronizationforcollectionsthathaveadefaultwaitForSyncvalueoftrue.
collection.updateByExample(document,newValue,keepNull,waitForSync,limit)
Theoptionallimitparametercanbeusedtorestrictthenumberofupdatestothespecifiedvalue.Iflimitisspecifiedbutlessthanthe
numberofdocumentsinthecollection,itisundefinedwhichdocumentsareupdated.
collection.updateByExample(document,newValue,options)
Usingthisvariant,theoptionsfortheoperationcanbepassedusinganobjectwiththefollowingsub-attributes:
keepNull
waitForSync
limit
mergeObjects
Examples
arangosh>db.example.save({Hello:"world",foo:"bar"});
arangosh>db.example.updateByExample({Hello:"world"},{Hello:"foo",World:"bar"},
false);
arangosh>db.example.byExample({Hello:"foo"}).toArray()
showexecutionresults
Collectiontype
collection.type()
Returnsthetypeofacollection.Possiblevaluesare:
2:documentcollection
3:edgecollection
GettheVersionofArangoDB
db._version()
72
CollectionM ethods
Returnstheserverversionstring.Notethatthisisnottheversionofthedatabase.
Examples
arangosh>require("@arangodb").db._version();
3.2.1
Edges
Edgesarenormaldocumentsthatalwayscontaina _fromanda _toattribute.Therefore,youcanusethedocumentmethodsto
operateonedges.Thefollowingmethods,however,arespecifictoedges.
edge-collection.edges(vertex)
Theedgesoperatorfindsalledgesstartingfrom(outbound)orendingin(inbound)vertex.
edge-collection.edges(vertices)
Theedgesoperatorfindsalledgesstartingfrom(outbound)orendingin(inbound)adocumentfromvertices,whichmustbealistof
documentsordocumenthandles.
arangosh>db._create("vertex");
arangosh>db._createEdgeCollection("relation");
arangosh>varmyGraph={};
arangosh>myGraph.v1=db.vertex.insert({name:"vertex1"});
arangosh>myGraph.v2=db.vertex.insert({name:"vertex2"});
arangosh>myGraph.e1=db.relation.insert(myGraph.v1,myGraph.v2,
........>{label:"knows"});
arangosh>db._document(myGraph.e1);
arangosh>db.relation.edges(myGraph.e1._id);
showexecutionresults
edge-collection.inEdges(vertex)
Theedgesoperatorfindsalledgesendingin(inbound)vertex.
edge-collection.inEdges(vertices)
Theedgesoperatorfindsalledgesendingin(inbound)adocumentfromvertices,whichmustalistofdocumentsordocumenthandles.
Examples
arangosh>db._create("vertex");
arangosh>db._createEdgeCollection("relation");
arangosh>myGraph.v1=db.vertex.insert({name:"vertex1"});
arangosh>myGraph.v2=db.vertex.insert({name:"vertex2"});
arangosh>myGraph.e1=db.relation.insert(myGraph.v1,myGraph.v2,
........>{label:"knows"});
arangosh>db._document(myGraph.e1);
arangosh>db.relation.inEdges(myGraph.v1._id);
arangosh>db.relation.inEdges(myGraph.v2._id);
showexecutionresults
edge-collection.outEdges(vertex)
Theedgesoperatorfindsalledgesstartingfrom(outbound)vertices.
edge-collection.outEdges(vertices)
Theedgesoperatorfindsalledgesstartingfrom(outbound)adocumentfromvertices,whichmustalistofdocumentsordocument
handles.
Examples
73
CollectionM ethods
arangosh>db._create("vertex");
arangosh>db._createEdgeCollection("relation");
arangosh>myGraph.v1=db.vertex.insert({name:"vertex1"});
arangosh>myGraph.v2=db.vertex.insert({name:"vertex2"});
arangosh>myGraph.e1=db.relation.insert(myGraph.v1,myGraph.v2,
........>{label:"knows"});
arangosh>db._document(myGraph.e1);
arangosh>db.relation.outEdges(myGraph.v1._id);
arangosh>db.relation.outEdges(myGraph.v2._id);
showexecutionresults
Misc
collection.iterate(iterator,options)
Iteratesoversomeelementsofthecollectionandapplythefunctioniteratortotheelements.Thefunctionwillbecalledwiththe
documentasfirstargumentandthecurrentnumber(startingwith0)assecondargument.
optionsmustbeanobjectwiththefollowingattributes:
limit(optional,defaultnone):useatmostlimitdocuments.
probability(optional,defaultall):anumberbetween0and1.Documentsarechosenwiththisprobability.
Examples
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.example.save({name:"Name/"+i+"/"+j,
........>home:[i,j],
........>work:[-i,-j]});
........>}
........>}
........>
arangosh>db.example.ensureIndex({type:"geo",fields:["home"]});
arangosh>items=db.example.getIndexes().map(function(x){returnx.id;});
........>db.example.index(items[1]);
showexecutionresults
74
DatabaseM ethods
DatabaseMethods
Document
db._document(object)
The_documentmethodfindsadocumentgivenanobjectobjectcontainingthe_idattribute.Themethodreturnsthedocumentifitcanbe
found.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.Anerrorisalsothrownifnodocument
existswiththegiven_id.
Pleasenotethatifthemethodisexecutedonthearangodserver(e.g.frominsideaFoxxapplication),animmutabledocumentobjectwill
bereturnedforperformancereasons.Itisnotpossibletochangeattributesofthisimmutableobject.Toupdateorpatchthereturned
document,itneedstobecloned/copiedintoaregularJavaScriptobjectfirst.Thisisnotnecessaryifthe_documentmethodiscalledfrom
outofarangoshorfromanyotherclient.
db._document(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.Norevisioncanbespecifiedinthiscase.
Examples
Returnsthedocument:
arangosh>db._document("example/12345");
showexecutionresults
Exists
db._exists(object)
The_existsmethoddetermineswhetheradocumentexistsgivenanobject objectcontainingthe_idattribute.
Anerroristhrownif_revisspecifiedbutthedocumentfoundhasadifferentrevisionalready.
Insteadofreturningthefounddocumentoranerror,thismethodwillonlyreturnanobjectwiththeattributes_id,_keyand_rev,orfalse
ifnodocumentwiththegiven_idor_keyexists.Itcanthusbeusedforeasyexistencechecks.
Thismethodwillthrowanerrorifusedimproperly,e.g.whencalledwithanon-documenthandle,anon-document,orwhenacrosscollectionrequestisperformed.
db._exists(document-handle)
Asbefore.Insteadofobjectadocument-handlecanbepassedasfirstargument.
Changesin3.0from2.8:
Inthecaseofarevisionmismatch_existsnowthrowsanerrorinsteadofsimplyreturningfalse.Thisistomakeitpossibletotellthe
differencebetweenarevisionmismatchandanon-existingdocument.
Replace
db._replace(selector,data)
Replacesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocument
withthat_idinthecurrentdatabase.Thisdocumentisthenreplacedwiththedatagivenassecondargument.Anyattribute_id,_keyor
_revindataisignored.
Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleofthe
updateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainsthe
revisionoftheold(nowreplaced)document.
75
DatabaseM ethods
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,
thereisaconflict,andanerroristhrown.
collection.replace(selector,data,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagis
beendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecific
operations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthe
collection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collectionsthathaveadefaultwaitForSyncvalueoftrue.
overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.
returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.
returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.
silent:Ifthisflagissettotrue,nooutputisreturned.
db._replace(document-handle,data)
db._replace(document-handle,data,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.
Examples
Createandreplaceadocument:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db._replace(a1,{a:2});
arangosh>a3=db._replace(a1,{a:3});
showexecutionresults
Changesin3.0from2.8:
Theoptionssilent,returnNewandreturnOldarenew.
Update
db._update(selector,data)
Updatesanexistingdocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocument
withthat_idinthecurrentdatabase.Thisdocumentisthenpatchedwiththedatagivenassecondargument.Anyattribute_id,_keyor
_revindataisignored.
Themethodreturnsadocumentwiththeattributes_id,_key,_revand_oldRev.Theattribute_idcontainsthedocumenthandleofthe
updateddocument,theattribute_revcontainsthedocumentrevisionoftheupdateddocument,theattribute_oldRevcontainsthe
revisionoftheold(nowupdated)document.
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,
thereisaconflict,andanerroristhrown.
db._update(selector,data,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagis
beendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecific
operations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthe
collection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collectionsthathaveadefaultwaitForSyncvalueoftrue.
overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.
returnNew:Ifthisflagissettotrue,thecompletenewdocumentisreturnedintheoutputundertheattributenew.
returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.
76
DatabaseM ethods
silent:Ifthisflagissettotrue,nooutputisreturned.
keepNull:TheoptionalkeepNullparametercanbeusedtomodifythebehaviorwhenhandlingnullvalues.Normally,nullvaluesare
storedinthedatabase.BysettingthekeepNullparametertofalse,thisbehaviorcanbechangedsothatallattributesindatawithnull
valueswillberemovedfromthetargetdocument.
mergeObjects:Controlswhetherobjects(notarrays)willbemergedifpresentinboththeexistingandthepatchdocument.Ifsetto
false,thevalueinthepatchdocumentwilloverwritetheexistingdocument'svalue.Ifsettotrue,objectswillbemerged.Thedefault
istrue.
db._update(document-handle,data)
db._update(document-handle,data,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisionpreconditionistested.
Examples
Createandupdateadocument:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db._update(a1,{b:2});
arangosh>a3=db._update(a1,{c:3});
showexecutionresults
Changesin3.0from2.8:
Theoptionssilent,returnNewandreturnOldarenew.
Remove
db._remove(selector)
Removesadocumentdescribedbytheselector,whichmustbeanobjectcontainingthe_idattribute.Theremustbeadocumentwiththat
_idinthecurrentdatabase.Thisdocumentisthenremoved.
Themethodreturnsadocumentwiththeattributes_id,_keyand_rev.Theattribute_idcontainsthedocumenthandleoftheremoved
document,theattribute_revcontainsthedocumentrevisionoftheremovedeocument.
Iftheselectorcontainsa_revattribute,themethodfirstchecksthatthespecifiedrevisionisthecurrentrevisionofthatdocument.Ifnot,
thereisaconflict,andanerroristhrown.
db._remove(selector,options)
Asbefore,butoptionsmustbeanobjectthatcancontainthefollowingbooleanattributes:
waitForSync:OnecanforcesynchronizationofthedocumentcreationoperationtodiskevenincasethatthewaitForSyncflagis
beendisabledfortheentirecollection.Thus,thewaitForSyncoptioncanbeusedtoforcesynchronizationofjustspecific
operations.Tousethis,setthewaitForSyncparametertotrue.IfthewaitForSyncparameterisnotspecifiedorsettofalse,thenthe
collection'sdefaultwaitForSyncbehaviorisapplied.ThewaitForSyncparametercannotbeusedtodisablesynchronizationfor
collectionsthathaveadefaultwaitForSyncvalueoftrue.
overwrite:Ifthisflagissettotrue,a_revattributeintheselectorisignored.
returnOld:Ifthisflagissettotrue,thecompletepreviousrevisionofthedocumentisreturnedintheoutputundertheattributeold.
silent:Ifthisflagissettotrue,nooutputisreturned.
db._remove(document-handle)
db._remove(document-handle,options)
Asbefore.Insteadofselectoradocument-handlecanbepassedasfirstargument.Norevisioncheckisperformed.
Examples
Removeadocument:
arangosh>a1=db.example.insert({a:1});
77
DatabaseM ethods
arangosh>db._remove(a1);
arangosh>db._remove(a1);
arangosh>db._remove(a1,{overwrite:true});
showexecutionresults
Removethedocumentintherevision a1withaconflict:
arangosh>a1=db.example.insert({a:1});
arangosh>a2=db._replace(a1,{a:2});
arangosh>db._remove(a1);
arangosh>db._remove(a1,{overwrite:true});
arangosh>db._document(a1);
showexecutionresults
Removeadocumentusingnewsignature:
arangosh>db.example.insert({_key:"11265325374",a:1});
arangosh>db.example.remove("example/11265325374",
........>{overwrite:true,waitForSync:false})
showexecutionresults
Changesin3.0from2.8:
Themethodnowreturnsnotonlytruebutinformationabouttheremoveddocument(s).TheoptionssilentandreturnOldarenew.
78
Graphs,Vertices&Edges
Graphs,Vertices&Edges
Graphs,vertices&edgesaredefinedintheGraphschapterindetails.
Relatedblogposts:
Graphsindatamodeling-istheemperornaked?
IndexFreeAdjacencyorHybridIndexesforGraphDatabases
79
NamingConventions
NamingConventionsinArangoDB
Thefollowingnamingconventionsshouldbefollowedbyuserswhencreatingdatabases,collectionsanddocumentsinArangoDB.
80
DatabaseNames
DatabaseNames
ArangoDBwillalwaysstartupwithadefaultdatabase,named_system.UserscancreateadditionaldatabasesinArangoDB,providedthe
databasenamesconformtothefollowingconstraints:
Databasenamesmustonlyconsistofthelettersatoz(bothloweranduppercaseallowed),thenumbers0to9,andtheunderscore
(_)ordash(-)symbolsThisalsomeansthatanynon-ASCIIdatabasenamesarenotallowed
Databasenamesmustalwaysstartwithaletter.Databasenamesstartingwithanunderscoreareconsideredtobesystemdatabases,
andusersshouldnotcreateordeletethose
Themaximumallowedlengthofadatabasenameis64bytes
Databasenamesarecase-sensitive
81
CollectionNames
CollectionNames
Userscanpicknamesfortheircollectionsasdesired,providedthefollowingnamingconstraintsarenotviolated:
Collectionnamesmustonlyconsistofthelettersatoz(bothinloweranduppercase),thenumbers0to9,andtheunderscore(_)
ordash(-)symbols.Thisalsomeansthatanynon-ASCIIcollectionnamesarenotallowed
User-definedcollectionnamesmustalwaysstartwithaletter.Systemcollectionnamesmuststartwithanunderscore.Allcollection
namesstartingwithanunderscoreareconsideredtobesystemcollectionsthatareforArangoDB'sinternaluseonly.System
collectionnamesshouldnotbeusedbyendusersfortheirowncollections
Themaximumallowedlengthofacollectionnameis64bytes
Collectionnamesarecase-sensitive
82
DocumentKeys
DocumentKeys
Userscandefinetheirownkeysfordocumentstheysave.Thedocumentkeywillbesavedalongwithadocumentinthe_keyattribute.
Userscanpickkeyvaluesasrequired,providedthatthevaluesconformtothefollowingrestrictions:
Thekeymustbeastringvalue.Numerickeysarenotallowed,butanynumericvaluecanbeputintoastringandcanthenbeusedas
documentkey.
Thekeymustbeatleast1byteandatmost254byteslong.Emptykeysaredisallowedwhenspecified(thoughitmaybevalidto
completelyomitthe_keyattributefromadocument)
Itmustconsistofthelettersa-z(loweroruppercase),thedigits0-9oranyofthefollowingpunctuationcharacters: _ - : .
@ ( ) + , = ; $ ! * ' %
Anyothercharacters,especiallymulti-byteUTF-8sequences,whitespaceorpunctuationcharacterscannotbeusedinsidekey
values
Thekeymustbeuniquewithinthecollectionitisused
Keysarecase-sensitive,i.e.myKeyandMyKEYareconsideredtobedifferentkeys.
Specifyingadocumentkeyisoptionalwhencreatingnewdocuments.Ifnodocumentkeyisspecifiedbytheuser,ArangoDBwillcreate
thedocumentkeyitselfaseachdocumentisrequiredtohaveakey.
Therearenoguaranteesabouttheformatandpatternofauto-generateddocumentkeysotherthantheaboverestrictions.Clientsshould
thereforetreatauto-generateddocumentkeysasopaquevaluesandnotrelyontheirformat.
Thecurrentformatforgeneratedkeysisastringcontainingnumericdigits.Thenumericvaluesreflectchronologicaltimeinthesensethat
_keyvaluesgeneratedlaterwillcontainhighernumbersthan_keyvaluesgeneratedearlier.Buttheexactvaluethatwillbegeneratedby
theserverisnotpredictable.Notethatifyousortonthe_keyattribute,stringcomparisonwillbeused,whichmeans "100"islessthan
"99"etc.
83
AttributeNames
AttributeNames
Userscanpickattributenamesfordocumentattributesasdesired,providedthefollowingattributenamingconstraintsarenotviolated:
AttributenamesstartingwithanunderscoreareconsideredtobesystemattributesforArangoDB'sinternaluse.Suchattribute
namesarealreadyusedbyArangoDBforspecialpurposes:
_idisusedtocontainadocument'shandle
_keyisusedtocontainadocument'suser-definedkey
_revisusedtocontainthedocument'srevisionnumber
Inedgecollections,the
_from
_to
attributesareusedtoreferenceotherdocuments.
M oresystemattributesmaybeaddedinthefuturewithoutfurthernoticesoendusersshouldtrytoavoidusingtheirownattribute
namesstartingwithunderscores.
Theoretically,attributenamescanincludepunctuationandspecialcharactersasdesired,providedthenameisavalidUTF-8string.
Formaximumportability,specialcharactersshouldbeavoidedthough.Forexample,attributenamesmaycontainthedotsymbol,
butthedothasaspecialmeaninginJavaScriptandalsoinAQL,sowhenusingsuchattributenamesinoneoftheselanguages,the
attributenameneedstobequotedbytheenduser.Overallitmightbebettertouseattributenameswhichdon'trequireany
quoting/escapinginalllanguagesused.Thisincludeslanguagesusedbytheclient(e.g.Ruby,PHP)iftheattributesaremappedto
objectmembersthere.
Attributenamesstartingwithanat-mark(@)willneedtobeenclosedinbacktickswhenusedinanAQLquerytotellthemapart
frombindvariables.Thereforewedonotencouragetheuseofattributesstartingwithat-marks,thoughtheywillworkwhenused
properly.
ArangoDBdoesnotenforcealengthlimitforattributenames.However,longattributenamesmayusemorememoryinresultsets
etc.Thereforetheuseoflongattributenamesisdiscouraged.
Attributenamesarecase-sensitive.
Attributeswithemptynames(anemptystring)aredisallowed.
84
Indexing
HandlingIndexes
ThisisanintroductiontoArangoDB'sinterfaceforindexesingeneral.
Therearespecialsectionsfor
IndexBasics:Introductiontoallindextypes
Whichindextousewhen:Indextypeandoptionsadviser
IndexUtilization:HowArangoDBusesindexes
WorkingwithIndexes:Howtohandleindexesprogrammaticallyusingthe dbobject
HashIndexes
Skiplists
PersistentIndexes
FulltextIndexes
Geo-spatialIndexes
Vertex-centricIndexes
85
IndexBasics
Indexbasics
Indexesallowfastaccesstodocuments,providedtheindexedattribute(s)areusedinaquery.WhileArangoDBautomaticallyindexes
somesystemattributes,usersarefreetocreateextraindexesonnon-systemattributesofdocuments.
User-definedindexescanbecreatedoncollectionlevel.M ostuser-definedindexescanbecreatedbyspecifyingthenamesoftheindex
attributes.Someindextypesallowindexingjustoneattribute(e.g.fulltextindex)whereasotherindextypesallowindexingmultiple
attributesatthesametime.
Thesystemattributes _id, _key, _fromand _toareautomaticallyindexedbyArangoDB,withouttheuserbeingrequiredtocreate
extraindexesforthem. _idand _keyarecoveredbyacollection'sprimarykey,and _fromand _toarecoveredbyanedge
collection'sedgeindexautomatically.
Usingthesystemattribute _idinuser-definedindexesisnotpossible,butindexing _key, _rev, _from,and _tois.
ArangoDBprovidesthefollowingindextypes:
PrimaryIndex
Foreachcollectiontherewillalwaysbeaprimaryindexwhichisahashindexforthedocumentkeys( _keyattribute)ofalldocuments
inthecollection.Theprimaryindexallowsquickselectionofdocumentsinthecollectionusingeitherthe _keyor _idattributes.It
willbeusedfromwithinAQLqueriesautomaticallywhenperformingequalitylookupson _keyor _id.
Therearealsodedicatedfunctionstofindadocumentgivenits _keyor _idthatwillalwaysmakeuseoftheprimaryindex:
db.collection.document("<document-key>");
db._document("<document-id>");
Astheprimaryindexisanunsortedhashindex,itcannotbeusedfornon-equalityrangequeriesorforsorting.
Theprimaryindexofacollectioncannotbedroppedorchanged,andthereisnomechanismtocreateuser-definedprimaryindexes.
EdgeIndex
Everyedgecollectionalsohasanautomaticallycreatededgeindex.Theedgeindexprovidesquickaccesstodocumentsbyeithertheir
_fromor _toattributes.Itcanthereforebeusedtoquicklyfindconnectionsbetweenvertexdocumentsandisinvokedwhenthe
connectingedgesofavertexarequeried.
EdgeindexesareusedfromwithinAQLwhenperformingequalitylookupson _fromor _tovaluesinanedgecollections.Thereare
alsodedicatedfunctionstofindedgesgiventheir _fromor _tovaluesthatwillalwaysmakeuseoftheedgeindex:
db.collection.edges("<from-value>");
db.collection.edges("<to-value>");
db.collection.outEdges("<from-value>");
db.collection.outEdges("<to-value>");
db.collection.inEdges("<from-value>");
db.collection.inEdges("<to-value>");
Internally,theedgeindexisimplementedasahashindex,whichstorestheunionofall _fromand _toattributes.Itcanbeusedfor
equalitylookups,butnotforrangequeriesorforsorting.Edgeindexesareautomaticallycreatedforedgecollections.Itisnotpossibleto
createuser-definededgeindexes.However,itispossibletofreelyusethe _fromand _toattributesinuser-definedindexes.
Anedgeindexcannotbedroppedorchanged.
HashIndex
Ahashindexcanbeusedtoquicklyfinddocumentswithspecificattributevalues.Thehashindexisunsorted,soitsupportsequality
lookupsbutnorangequeriesorsorting.
86
IndexBasics
Ahashindexcanbecreatedononeormultipledocumentattributes.Ahashindexwillonlybeusedbyaqueryifallindexattributesare
presentinthesearchcondition,andifallattributesarecomparedusingtheequality( ==)operator.Hashindexesareusedfromwithin
AQLandseveralqueryfunctions,e.g. byExample, firstExampleetc.
Hashindexescanoptionallybedeclaredunique,thendisallowingsavingthesamevalue(s)intheindexedattribute(s).Hashindexescan
optionallybesparse.
Thedifferenttypesofhashindexeshavethefollowingcharacteristics:
uniquehashindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.
Tryingtoinsertadocumentwiththesamekeyvalueasanalreadyexistingdocumentwillleadtoauniqueconstraintviolation.
Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueof nullintheindex
attribute(s)willstillbeindexed.Akeyvalueof nullmayonlyoccuronceintheindex,sothistypeofindexcannotbeusedfor
optionalattributes.
Theuniqueoptioncanalsobeusedtoensurethatnoduplicateedgesarecreated,byaddingacombinedindexforthefields _from
and _totoanedgecollection.
unique,sparsehashindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheunique
index.Documentsinwhichatleastoneoftheindexattributesisnotsetorhasavalueof nullarenotincludedintheindex.This
typeofindexcanbeusedtoensurethattherearenoduplicatekeysinthecollectionfordocumentswhichhavetheindexed
attributesset.Astheindexwillexcludedocumentsforwhichtheindexedattributesare nullornotset,itcanbeusedforoptional
attributes.
non-uniquehashindex:alldocumentsinthecollectionwillbeindexed.Thistypeofindexisnotsparse.Documentsthatdonot
containtheindexattributesorthathaveavalueof nullintheindexattribute(s)willstillbeindexed.Duplicatekeyvaluescan
occuranddonotleadtouniqueconstraintviolations.
non-unique,sparsehashindex:onlythosedocumentswillbeindexedthathavealltheindexedattributessettoavalueotherthan
null.Itcanbeusedforoptionalattributes.
Theamortizedcomplexityoflookup,insert,update,andremovaloperationsinuniquehashindexesisO(1).
Non-uniquehashindexeshaveanamortizedcomplexityofO(1)forinsert,update,andremovaloperations.Thatmeansnon-uniquehash
indexescanbeusedonattributeswithlowcardinality.
Ifahashindexiscreatedonanattributethatismissinginallormanyofthedocuments,thebehaviorisasfollows:
iftheindexissparse,thedocumentsmissingtheattributewillnotbeindexedandnotuseindexmemory.Thesedocumentswillnot
influencetheupdateorremovalperformancefortheindex.
iftheindexisnon-sparse,thedocumentsmissingtheattributewillbecontainedintheindexwithakeyvalueof null.
Hashindexessupportindexingarrayvaluesiftheindexattributenameisextendedwitha[*].
SkiplistIndex
Askiplistisasortedindexstructure.Itcanbeusedtoquicklyfinddocumentswithspecificattributevalues,forrangequeriesandfor
returningdocumentsfromtheindexinsortedorder.SkiplistswillbeusedfromwithinAQLandseveralqueryfunctions,e.g. byExample,
firstExampleetc.
Skiplistindexeswillbeusedforlookups,rangequeriesandsortingonlyifeitherallindexattributesareprovidedinaquery,orifa
leftmostprefixoftheindexattributesisspecified.
Forexample,ifaskiplistindexiscreatedonattributes value1and value2,thefollowingfilterconditionscanusetheindex(note:the
<=and >=operatorsareintentionallyomittedhereforthesakeofbrevity):
FILTERdoc.value1==...
FILTERdoc.value1<...
FILTERdoc.value1>...
FILTERdoc.value1>...&&doc.value1<...
FILTERdoc.value1==...&&doc.value2==...
FILTERdoc.value1==...&&doc.value2>...
FILTERdoc.value1==...&&doc.value2>...&&doc.value2<...
87
IndexBasics
Inordertouseaskiplistindexforsorting,theindexattributesmustbespecifiedinthe SORTclauseofthequeryinthesameorderas
theyappearintheindexdefinition.Skiplistindexesarealwayscreatedinascendingorder,buttheycanbeusedtoaccesstheindexed
elementsinbothascendingordescendingorder.However,foracombinedindex(anindexonmultipleattributes)thisrequiresthatthesort
ordersinasinglequeryasspecifiedinthe SORTclausemustbeeitherallascending(optionallyommittedasascendingisthedefault)or
alldescending.
Forexample,iftheskiplistindexiscreatedonattributes value1and value2(inthisorder),thenthefollowingsortsclausescanuse
theindexforsorting:
SORTvalue1ASC,value2ASC(anditsequivalent SORTvalue1,value2)
SORTvalue1DESC,value2DESC
SORTvalue1ASC(anditsequivalent SORTvalue1)
SORTvalue1DESC
Thefollowingsortclausescannotmakeuseoftheindexorder,andrequireanextrasortstep:
SORTvalue1ASC,value2DESC
SORTvalue1DESC,value2ASC
SORTvalue2(anditsequivalent SORTvalue2ASC)
SORTvalue2DESC(becausefirstindexedattribute value1isnotusedinsortclause)
Note:thelattertwosortclausescannotusetheindexbecausethesortclausedoesnotrefertoaleftmostprefixoftheindexattributes.
Skiplistscanoptionallybedeclaredunique,disallowingsavingthesamevalueintheindexedattribute.Theycanbesparseornon-sparse.
Thedifferenttypesofskiplistindexeshavethefollowingcharacteristics:
uniqueskiplistindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheuniqueindex.
Tryingtoinsertadocumentwiththesamekeyvalueasanalreadyexistingdocumentwillleadtoauniqueconstraintviolation.
Thistypeofindexisnotsparse.Documentsthatdonotcontaintheindexattributesorthathaveavalueof nullintheindex
attribute(s)willstillbeindexed.Akeyvalueof nullmayonlyoccuronceintheindex,sothistypeofindexcannotbeusedfor
optionalattributes.
unique,sparseskiplistindex:alldocumentsinthecollectionmusthavedifferentvaluesfortheattributescoveredbytheunique
index.Documentsinwhichatleastoneoftheindexattributesisnotsetorhasavalueof nullarenotincludedintheindex.This
typeofindexcanbeusedtoensurethattherearenoduplicatekeysinthecollectionfordocumentswhichhavetheindexed
attributesset.Astheindexwillexcludedocumentsforwhichtheindexedattributesare nullornotset,itcanbeusedforoptional
attributes.
non-uniqueskiplistindex:alldocumentsinthecollectionwillbeindexed.Thistypeofindexisnotsparse.Documentsthatdonot
containtheindexattributesorthathaveavalueof nullintheindexattribute(s)willstillbeindexed.Duplicatekeyvaluescan
occuranddonotleadtouniqueconstraintviolations.
non-unique,sparseskiplistindex:onlythosedocumentswillbeindexedthathavealltheindexedattributessettoavalueother
than null.Itcanbeusedforoptionalattributes.
Theoperationalamortizedcomplexityforskiplistindexesislogarithmicallycorrelatedwiththenumberofdocumentsintheindex.
Skiplistindexessupportindexingarrayvaluesiftheindexattributenameisextendedwitha[*]`.
PersistentIndex
Thepersistentindexisasortedindexwithpersistence.Theindexentriesarewrittentodiskwhendocumentsarestoredorupdated.That
meanstheindexentriesdonotneedtoberebuiltfromthecollectiondatawhentheserverisrestartedortheindexedcollectionisinitially
loaded.Thususingpersistentindexesmayreducecollectionloadingtimes.
Thepersistentindextypecanbeusedforsecondaryindexesatthemoment.Thatmeansthepersistentindexcurrentlycannotbemade
theonlyindexforacollection,becausetherewillalwaysbethein-memoryprimaryindexforthecollectioninaddition,andpotentially
moreindexes(suchastheedgesindexforanedgecollection).
TheindeximplementationisusingtheRocksDBengine,anditprovideslogarithmiccomplexityforinsert,update,andremoveoperations.
Asthepersistentindexisnotanin-memoryindex,itdoesnotstorepointersintotheprimaryindexasallthein-memoryindexesdo,but
insteaditstoresadocument'sprimarykey.Toretrieveadocumentviaapersistentindexviaanindexvaluelookup,therewillthereforebe
88
IndexBasics
anadditionalO(1)lookupintotheprimaryindextofetchtheactualdocument.
Asthepersistentindexissorted,itcanbeusedforpointlookups,rangequeriesandsortingoperations,butonlyifeitherallindex
attributesareprovidedinaquery,orifaleftmostprefixoftheindexattributesisspecified.
GeoIndex
Userscancreateadditionalgeoindexesononeormultipleattributesincollections.Ageoindexisusedtofindplacesonthesurfaceofthe
earthfast.
Thegeoindexstorestwo-dimensionalcoordinates.Itcanbecreatedoneithertwoseparatedocumentattributes(latitudeandlongitude)or
asinglearrayattributethatcontainsbothlatitudeandlongitude.Latitudeandlongitudemustbenumericvalues.
Thegeoindexprovidesoperationstofinddocumentswithcoordinatesnearesttoagivencomparisoncoordinate,andtofinddocuments
withcoordinatesthatarewithinaspecifiableradiusaroundacomparisoncoordinate.
ThegeoindexisusedviadedicatedfunctionsinAQL,thesimplequeriesfunctionsanditisimplicitlyappliedwheninAQLaSORTor
FILTERisusedwiththedistancefunction.Otherwiseitwillnotbeusedforothertypesofqueriesorconditions.
FulltextIndex
Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.Afulltextindexcanbecreatedonasingleattribute
only,andwillindexallwordscontainedindocumentsthathaveatextualvalueinthatattribute.Onlywordswitha(specifiable)minimum
lengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedbylibicu,whichistakingintoaccountthe
selectedlanguageprovidedatserverstart.Wordsareindexedintheirlower-casedform.Theindexsupportscompletematchqueries(full
words)andprefixqueries,plusbasiclogicaloperationssuchas and, orand notforcombiningpartialresults.
Thefulltextindexissparse,meaningitwillonlyindexdocumentsforwhichtheindexattributeissetandcontainsastringvalue.
Additionally,onlywordswithaconfigurableminimumlengthwillbeincludedintheindex.
ThefulltextindexisusedviadedicatedfunctionsinAQLorthesimplequeries,butwillnotbeenabledforothertypesofqueriesor
conditions.
Indexingattributesandsub-attributes
Top-levelaswellasnestedattributescanbeindexed.Forattributesatthetoplevel,theattributenamesalonearerequired.Toindexa
singlefield,passanarraywithasingleelement(stringoftheattributekey)tothefieldsparameteroftheensureIndex()method.Tocreate
acombinedindexovermultiplefields,simplyaddmorememberstothefieldsarray:
//{name:"Smith",age:35}
db.posts.ensureIndex({type:"hash",fields:["name"]})
db.posts.ensureIndex({type:"hash",fields:["name","age"]})
Toindexsub-attributes,specifytheattributepathusingthedotnotation:
//{name:{last:"Smith",first:"John"}}
db.posts.ensureIndex({type:"hash",fields:["name.last"]})
db.posts.ensureIndex({type:"hash",fields:["name.last","name.first"]})
Indexingarrayvalues
Ifanindexattributecontainsanarray,ArangoDBwillstoretheentirearrayastheindexvaluebydefault.Accessingindividualmembers
ofthearrayviatheindexisnotpossiblethisway.
Tomakeanindexinserttheindividualarraymembersintotheindexinsteadoftheentirearrayvalue,aspecialarrayindexneedstobe
createdfortheattribute.Arrayindexescanbesetuplikeregularhashorskiplistindexesusingthe collection.ensureIndex()function.
Tomakeahashorskiplistindexanarrayindex,theindexattributenameneedstobeextendedwith[*]whencreatingtheindexandwhen
filteringinanAQLqueryusingthe INoperator.
Thefollowingexamplecreatesanarrayhashindexonthe tagsattributeinacollectionnamed posts:
89
IndexBasics
db.posts.ensureIndex({type:"hash",fields:["tags[*]"]});
db.posts.insert({tags:["foobar","baz","quux"]});
Thisarrayindexcanthenbeusedforlookingupindividual tagsvaluesfromAQLqueriesviathe INoperator:
FORdocINposts
FILTER'foobar'INdoc.tags
RETURNdoc
Itispossibletoaddthearrayexpansionoperator[*],butitisnotmandatory.Youmayuseittoindicatethatanarrayindexisused,itis
purelycosmetichowever:
FORdocINposts
FILTER'foobar'INdoc.tags[*]
RETURNdoc
ThefollowingFILTERconditionswillnotusethearrayindex:
FILTERdoc.tagsANY=='foobar'
FILTERdoc.tagsANYIN'foobar'
FILTERdoc.tagsIN'foobar'
FILTERdoc.tags=='foobar'
FILTER'foobar'==doc.tags
Itisalsopossibletocreateanindexonsubattributesofarrayvalues.Thismakessenseiftheindexattributeisanarrayofobjects,e.g.
db.posts.ensureIndex({type:"hash",fields:["tags[*].name"]});
db.posts.insert({tags:[{name:"foobar"},{name:"baz"},{name:"quux"}]});
Thefollowingquerywillthenusethearrayindex(thisdoesrequirethearrayexpansionoperator):
FORdocINposts
FILTER'foobar'INdoc.tags[*].name
RETURNdoc
Ifyoustoreadocumenthavingthearraywhichdoescontainelementsnothavingthesubattributesthisdocumentwillalsobeindexed
withthevalue null,whichinArangoDBisequaltoattributenotexisting.
ArangoDBsupportscreatingarrayindexeswithasingle[*]operatorperindexattribute.Forexample,creatinganindexasfollowsisnot
supported:
db.posts.ensureIndex({type:"hash",fields:["tags[*].name[*].value"]});
Arrayvalueswillautomaticallybede-duplicatedbeforebeinginsertedintoanarrayindex.Forexample,ifthefollowingdocumentis
insertedintothecollection,theduplicatearrayvalue barwillbeinsertedonlyonce:
db.posts.insert({tags:["foobar","bar","bar"]});
Thisisdonetoavoidredudantstorageofthesameindexvalueforthesamedocument,whichwouldnotprovideanybenefit.
Ifanarrayindexisdeclaredunique,thede-duplicationofarrayvalueswillhappenbeforeinsertingthevaluesintotheindex,sotheabove
insertoperationwithtwoidenticalvalues barwillnotnecessarilyfail
Itwillalwaysfailiftheindexalreadycontainsaninstanceofthe barvalue.However,ifthevalue barisnotalreadypresentinthe
index,thenthede-duplicationofthearrayvalueswilleffectivelyleadto barbeinginsertedonlyonce.
Toturnoffthededuplicationofarrayvalues,itispossibletosetthededuplicateattributeonthearrayindexto false.Thedefault
valuefordeduplicateis truehowever,sode-duplicationwilltakeplaceifnotexplicitlyturnedoff.
db.posts.ensureIndex({type:"hash",fields:["tags[*]"],deduplicate:false});
90
IndexBasics
//willfailnow
db.posts.insert({tags:["foobar","bar","bar"]});
Ifanarrayindexisdeclaredandyoustoredocumentsthatdonothaveanarrayatthespecifiedattributethisdocumentwillnotbe
insertedintheindex.Hencethefollowingobjectswillnotbeindexed:
db.posts.ensureIndex({type:"hash",fields:["tags[*]"]});
db.posts.insert({something:"else"});
db.posts.insert({tags:null});
db.posts.insert({tags:"thisisnoarray"});
db.posts.insert({tags:{content:[1,2,3]}});
Anarrayindexisabletoindexexplicit nullvalues.Whenqueriedfor nullvalues,itwillonlyreturnthosedocumentshaving
explicitly nullstoredinthearray,itwillnotreturnanydocumentsthatdonothavethearrayatall.
db.posts.ensureIndex({type:"hash",fields:["tags[*]"]});
db.posts.insert({tags:null})//Willnotbeindexed
db.posts.insert({tags:[]})//Willnotbeindexed
db.posts.insert({tags:[null]});//Willbeindexedfornull
db.posts.insert({tags:[null,1,2]});//Willbeindexedfornull,1and2
Declaringanarrayindexassparsedoesnothaveaneffectonthearraypartoftheindex,thisinparticularmeansthatexplicit null
valuesarealsoindexedinthesparseversion.Ifanindexiscombinedfromanarrayandanormalattributethesparsitywillapplyforthe
attributee.g.:
db.posts.ensureIndex({type:"hash",fields:["tags[*]","name"],sparse:true});
db.posts.insert({tags:null,name:"alice"})//Willnotbeindexed
db.posts.insert({tags:[],name:"alice"})//Willnotbeindexed
db.posts.insert({tags:[1,2,3]})//Willnotbeindexed
db.posts.insert({tags:[1,2,3],name:null})//Willnotbeindexed
db.posts.insert({tags:[1,2,3],name:"alice"})
//Willbeindexedfor[1,"alice"],[2,"alice"],[3,"alice"]
db.posts.insert({tags:[null],name:"bob"})
//Willbeindexedfor[null,"bob"]
PleasenotethatfilteringusingarrayindexesonlyworksfromwithinAQLqueriesandonlyifthequeryfiltersontheindexedattribute
usingthe INoperator.Theothercomparisonoperators( ==, !=, >, >=, <, <=, ANY, ALL, NONE)currentlycannotuse
arrayindexes.
Vertexcentricindexes
Asmentionedabove,themostimportantindexesforgraphsaretheedgeindexes,indexingthe _fromand _toattributesofedge
collections.Theyprovideveryquickaccesstoalledgesoriginatinginorarrivingatagivenvertex,whichallowstoquicklyfindall
neighboursofavertexinagraph.
Inmanycasesonewouldliketorunmorespecificqueries,forexamplefindingamongsttheedgesoriginatinginagivenvertexonlythose
withthe20latesttimestamps.Exactlythisisachievedwith"vertexcentricindexes".Inasensethesearelocalizedindexesforanedge
collection,whichsitateverysinglevertex.
Technically,theyareimplementedinArangoDBasindexes,whichsortthecompleteedgecollectionfirstby _fromandthenbyother
attributes.Ifweforexamplehaveaskiplistindexontheattributes _fromand timestampofanedgecollection,wecananswerthe
abovequestionveryquicklywithasinglerangelookupintheindex.
SinceArangoDB3.0onecancreatesortedindexes(type"skiplist"and"persistent")thatindexthespecialedgeattributes _fromor
_toandadditionallyotherattributes.SinceArangoDB3.1,theseareusedingraphtraversals,whenappropriate FILTERstatements
arefoundbytheoptimizer.
Forexample,tocreateavertexcentricindexoftheabovetype,youwouldsimplydo
db.edges.ensureIndex({"type":"skiplist","fields":["_from","timestamp"]});
Then,querieslike
91
IndexBasics
FORv,e,pIN1..1OUTBOUND"V/1"edges
FILTERe.timestampALL>="2016-11-09"
RETURNp
willbeconsiderablyfasterincasetherearemanyedgesoriginatinginvertex "V/1"butonlyfewwitharecenttimestamp.
92
Whichindextousewhen
WhichIndextousewhen
ArangoDBautomaticallyindexesthe _keyattributeineachcollection.Thereisnoneedtoindexthisattributeseparately.Pleasenote
thatadocument's _idattributeisderivedfromthe _keyattribute,andisthusimplicitlyindexed,too.
ArangoDBwillalsoautomaticallycreateanindexon _fromand _toinanyedgecollection,meaningincomingandoutgoing
connectionscanbedeterminedefficiently.
Indextypes
Userscandefineadditionalindexesononeormultipledocumentattributes.SeveraldifferentindextypesareprovidedbyArangoDB.
Theseindexeshavedifferentusagescenarios:
hashindex:providesquickaccesstoindividualdocumentsif(andonlyif)allindexedattributesareprovidedinthesearchquery.The
indexwillonlybeusedforequalitycomparisons.Itdoesnotsupportrangequeriesandcannotbeusedforsorting.
Thehashindexisagoodcandidateifallormostqueriesontheindexedattribute(s)areequalitycomparisons.Theuniquehashindex
providesanamortizedcomplexityofO(1)forinsert,update,removeandlookupoperations.Thenon-uniquehashindexprovides
O(1)inserts,updatesandremoves,andwillallowlookingupdocumentsbyindexvaluewithamortizedO(n)complexity,withn
beingthenumberofdocumentswiththatindexvalue.
Anon-uniquehashindexonanoptionaldocumentattributeshouldbedeclaredsparsesothatitwillnotindexdocumentsforwhich
theindexattributeisnotset.
skiplistindex:skiplistskeeptheindexedvaluesinanorder,sotheycanbeusedforequalitylookups,rangequeriesandforsorting.
Forhighselectivityattributes,skiplistindexeswillhaveahigheroverheadthanhashindexes.Forlowselectivityattributes,skiplist
indexeswillbemoreefficientthannon-uniquehashindexes.
Additionally,skiplistindexesallowmoreusecases(e.g.rangequeries,sorting)thanhashindexes.Furthermore,theycanbeusedfor
lookupsbasedonaleftmostprefixoftheindexattributes.
persistentindex:apersistentindexbehavesmuchlikethesortedskiplistindex,exceptthatallindexvaluesarepersistedondiskand
donotneedtoberebuiltinmemorywhentheserverisrestartedortheindexedcollectionisreloaded.Theoperationsinapersistent
indexhavelogarithmiccomplexity,butoperationshavemayhaveahigherconstantfactorthantheoperationsinaskiplistindex,
becausethepersistentindexmayneedtomakeextraroundtripstotheprimaryindextofetchtheactualdocuments.
Apersistentindexcanbeusedforequalitylookups,rangequeriesandforsorting.Forhighselectivityattributes,persistentindexes
willhaveahigheroverheadthanskiplistorhashindexes.
Persistentindexesallowmoreusecases(e.g.rangequeries,sorting)thanhashindexes.Furthermore,theycanbeusedforlookups
basedonaleftmostprefixoftheindexattributes.Incontrasttothein-memoryskiplistindexes,persistentindexesdonotneedtobe
rebuiltin-memorysotheydon'tinfluencetheloadingtimeofcollectionsasotherin-memoryindexesdo.
geoindex:thegeoindexprovidedbyArangoDBallowssearchingfordocumentswithinaradiusaroundatwo-dimensionalearth
coordinate(point),ortofinddocumentswithareclosesttoapoint.Documentcoordinatescaneitherbespecifiedintwodifferent
documentattributesorinasingleattribute,e.g.
{"latitude":50.9406645,"longitude":6.9599115}
or
{"coords":[50.9406645,6.9599115]}
GeoindexeswillbeinvokedviaspecialfunctionsorAQLoptimization.Theoptimizationcanbetriggeredwhenacollectionwith
geoindexisenumeratedandaSORTorFILTERstatementisusedinconjunctionwiththedistancefunction.
fulltextindex:afulltextindexcanbeusedtoindexallwordscontainedinaspecificattributeofalldocumentsinacollection.Only
wordswitha(specifiable)minimumlengthareindexed.Wordtokenizationisdoneusingthewordboundaryanalysisprovidedby
libicu,whichistakingintoaccounttheselectedlanguageprovidedatserverstart.
93
Whichindextousewhen
Theindexsupportscompletematchqueries(fullwords)andprefixqueries.Fulltextindexeswillonlybeinvokedviaspecial
functions.
Sparsevs.non-sparseindexes
Hashindexesandskiplistindexescanoptionallybecreatedsparse.Asparseindexdoesnotcontaindocumentsforwhichatleastoneof
theindexattributeisnotsetorcontainsavalueof null.
Assuchdocumentsareexcludedfromsparseindexes,theymaycontainfewerdocumentsthantheirnon-sparsecounterparts.This
enablesfasterindexingandcanleadtoreducedmemoryusageincasetheindexedattributedoesoccuronlyinsome,butnotalldocuments
ofthecollection.Sparseindexeswillalsoreducethenumberofcollisionsinnon-uniquehashindexesincasenon-existingoroptional
attributesareindexed.
Inordertocreateasparseindex,anobjectwiththeattribute sparsecanbeaddedtotheindexcreationcommands:
db.collection.ensureIndex({type:"hash",fields:["attributeName"],sparse:true});
db.collection.ensureIndex({type:"hash",fields:["attributeName1","attributeName2"],sparse:true});
db.collection.ensureIndex({type:"hash",fields:["attributeName"],unique:true,sparse:true});
db.collection.ensureIndex({type:"hash",fields:["attributeName1","attributeName2"],unique:true,sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName"],sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName1","attributeName2"],sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName"],unique:true,sparse:true});
db.collection.ensureIndex({type:"skiplist",fields:["attributeName1","attributeName2"],unique:true,sparse:true});
Whennotexplicitlyset,the sparseattributedefaultsto falsefornewindexes.Otherindexesthanhashandskiplistdonotsupport
sparsity.
Assparseindexesmayexcludesomedocumentsfromthecollection,theycannotbeusedforalltypesofqueries.Sparsehashindexes
cannotbeusedtofinddocumentsforwhichatleastoneoftheindexedattributeshasavalueof null.Forexample,thefollowingAQL
querycannotuseasparseindex,evenifonewascreatedonattribute attr:
FORdocIncollection
FILTERdoc.attr==null
RETURNdoc
Ifthelookupvalueisnon-constant,asparseindexmayormaynotbeused,dependingontheothertypesofconditionsinthequery.If
theoptimizercansafelydeterminethatthelookupvaluecannotbe null,asparseindexmaybeused.Whenuncertain,theoptimizer
willnotmakeuseofasparseindexinaqueryinordertoproducecorrectresults.
Forexample,thefollowingqueriescannotuseasparseindexon attrbecausetheoptimizerwillnotknowbeforehandwhetherthe
valueswhicharecomparedto doc.attrwillinclude null:
FORdocIncollection
FILTERdoc.attr==SOME_FUNCTION(...)
RETURNdoc
FORotherINotherCollection
FORdocIncollection
FILTERdoc.attr==other.attr
RETURNdoc
Sparseskiplistindexescanbeusedforsortingiftheoptimizercansafelydetectthattheindexrangedoesnotinclude nullforanyof
theindexattributes.
Notethatifyouintendtousejoinsitmaybeclevertousenon-sparsityandmaybeevenuniquenessforthatattribute,elseallitems
containingthe nullvaluewillmatchagainsteachotherandthusproducelargeresults.
94
IndexUtilization
IndexUtilization
InmostcasesArangoDBwilluseasingleindexpercollectioninagivenquery.AQLqueriescanusemorethanoneindexpercollection
whenmultipleFILTERconditionsarecombinedwithalogical ORandthesecanbecoveredbyindexes.AQLquerieswilluseasingle
indexpercollectionwhenFILTERconditionsarecombinedwithlogical AND.
Creatingmultipleindexesondifferentattributesofthesamecollectionmaygivethequeryoptimizermorechoiceswhenpickinganindex.
Creatingmultipleindexesondifferentattributescanalsohelpinspeedingupdifferentqueries,withFILTERconditionsondifferent
attributes.
Itisoftenbeneficialtocreateanindexonmorethanjustoneattribute.Byaddingmoreattributestoanindex,anindexcanbecomemore
selectiveandthusreducethenumberofdocumentsthatqueriesneedtoprocess.
ArangoDB'sprimaryindexes,edgesindexesandhashindexeswillautomaticallyprovideselectivityestimates.Indexselectivityestimates
areprovidedinthewebinterface,the getIndexes()returnvalueandinthe explain()outputforagivenquery.
Themoreselectiveanindexis,themoredocumentsitwillfilteronaverage.Theindexselectivityestimatesarethereforeusedbythe
optimizerwhencreatingqueryexecutionplanswhentherearemultipleindexestheoptimizercanchoosefrom.Theoptimizerwillthen
selectacombinationofindexeswiththelowestestimatedtotalcost.Ingeneral,theoptimizerwillpicktheindexeswiththehighest
estimatedselectivity.
Sparseindexesmayormaynotbepickedbytheoptimizerinaquery.Assparseindexesdonotcontain nullvalues,theywillnotbe
usedforqueriesiftheoptimizercannotsafelydeterminewhetheraFILTERconditionincludes nullvaluesfortheindexattributes.The
optimizerpolicyistoproducecorrectresults,regardlessofwhetherorwhichindexisusedtosatisfyFILTERconditions.Ifitisunsure
aboutwhetherusinganindexwillviolatethepolicy,itwillnotmakeuseoftheindex.
Troubleshooting
WhenindoubtaboutwhetherandwhichindexeswillbeusedforexecutingagivenAQLquery,clicktheExplainbuttonintheweb
interfaceintheQueriesvieworusethe explain()methodforthestatementasfollows(fromtheArangoShell):
varquery="FORdocINcollectionFILTERdoc.value>42RETURNdoc";
varstmt=db._createStatement(query);
stmt.explain();
The explain()commandwillreturnadetailedJSONrepresentationofthequery'sexecutionplan.TheJSONexplainoutputisintended
tobeusedbycode.Togetahuman-readableandmuchmorecompactexplanationofthequery,thereisanexplainertool:
varquery="FORdocINcollectionFILTERdoc.value>42RETURNdoc";
require("@arangodb/aql/explainer").explain(query);
Ifanyoftheexplainmethodsshowsthataqueryisnotusingindexes,thefollowingstepsmayhelp:
checkiftheattributenamesinthequeryarecorrectlyspelled.Inaschema-freedatabase,documentsinthesamecollectioncanhave
varyingstructures.Thereisnosuchthingasanon-existingattributeerror.Aquerythatreferstoattributenamesnotpresentinany
ofthedocumentswillnotreturnanerror,andobviouslywillnotbenefitfromindexes.
checkthereturnvalueofthe getIndexes()methodforthecollectionsusedinthequeryandvalidatethatindexesareactually
presentontheattributesusedinthequery'sfilterconditions.
ifindexesarepresentbutnotusedbythequery,theindexesmayhavethewrongtype.Forexample,ahashindexwillonlybeused
forequalitycomparisons(i.e. ==)butnotforothercomparisontypessuchas <, <=, >, >=.Additionallyhashindexeswill
onlybeusedifalloftheindexattributesareusedinthequery'sFILTERconditions.Askiplistindexwillonlybeusedifatleastits
firstattributeisusedinaFILTERcondition.Ifadditionallyoftheskiplistindexattributesarespecifiedinthequery(fromleft-toright),theymayalsobeusedandallowtofiltermoredocuments.
usingindexedattributesasfunctionparametersorinarbitraryexpressionswilllikelyleadtotheindexontheattributenotbeing
used.Forexample,thefollowingquerieswillnotuseanindexon value:
95
IndexUtilization
FORdocINcollectionFILTERTO_NUMBER(doc.value)==42RETURNdoc
FORdocINcollectionFILTERdoc.value-1==42RETURNdoc
Inthesecasesthequeriesshouldberewrittensothatonlytheindexattributeispresentononesideoftheoperator,oradditional
filtersandindexesshouldbeusedtorestricttheamountofdocumentsotherwise.
certainAQLfunctionssuchas WITHIN()or FULLTEXT()doutilizeindexesinternally,buttheiruseisnotmentionedinthequery
explanationforfunctionsingeneral.Thesefunctionswillraisequeryerrors(atruntime)ifnosuitableindexispresentforthe
collectioninquestion.
thequeryoptimizerwillingeneralpickoneindexpercollectioninaquery.Itcanpickmorethanoneindexpercollectionifthe
FILTERconditioncontainsmultiplebranchescombinedwithlogical OR.Forexample,thefollowingqueriescanuseindexes:
FORdocINcollectionFILTERdoc.value1==42||doc.value1==23RETURNdoc
FORdocINcollectionFILTERdoc.value1==42||doc.value2==23RETURNdoc
FORdocINcollectionFILTERdoc.value1<42||doc.value2>23RETURNdoc
Thetwo ORsinthefirstquerywillbeconvertedtoan INlist,andifthereisasuitableindexon value1,itwillbeused.The
secondqueryrequirestwoseparateindexeson value1and value2andwillusethemifpresent.Thethirdquerycanusethe
indexeson value1and value2whentheyaresorted.
96
WorkingwithIndexes
WorkingwithIndexes
IndexIdentifiersandHandles
Anindexhandleuniquelyidentifiesanindexinthedatabase.Itisastringandconsistsofthecollectionnameandanindexidentifier
separatedbya /.Theindexidentifierpartisanumericvaluethatisauto-generatedbyArangoDB.
Aspecificindexofacollectioncanbeaccessedusingitsindexhandleorindexidentifierasfollows:
db.collection.index("<index-handle>");
db.collection.index("<index-identifier>");
db._index("<index-handle>");
Forexample:Assumethattheindexhandle,whichisstoredinthe _idattributeoftheindex,is demo/362549736andtheindexwas
createdinacollectionnamed demo.Thenthisindexcanbeaccessedas:
db.demo.index("demo/362549736");
Becausetheindexhandleisuniquewithinthedatabase,youcanleaveoutthecollectionandusetheshortcut:
db._index("demo/362549736");
CollectionMethods
Listingallindexesofacollection
returnsinformationabouttheindexes getIndexes()
Returnsanarrayofallindexesdefinedforthecollection.
Notethat _keyimplicitlyhasanindexassignedtoit.
arangosh>db.test.ensureHashIndex("hashListAttribute",
........>"hashListSecondAttribute.subAttribute");
arangosh>db.test.getIndexes();
showexecutionresults
Creatinganindex
IndexesshouldbecreatedusingthegeneralmethodensureIndex.Thismethodobsoletesthespecializedindex-specificmethods
ensureHashIndex,ensureSkiplist,ensureUniqueConstraintetc.
ensuresthatanindexexists collection.ensureIndex(index-description)
Ensuresthatanindexaccordingtotheindex-descriptionexists.Anewindexwillbecreatedifnoneexistswiththegivendescription.
Theindex-descriptionmustcontainatleastatypeattribute.Otherattributesmaybenecessary,dependingontheindextype.
typecanbeoneofthefollowingvalues:
hash:hashindex
skiplist:skiplistindex
fulltext:fulltextindex
geo1:geoindex,withoneattribute
geo2:geoindex,withtwoattributes
97
WorkingwithIndexes
sparsecanbetrueorfalse.
Forhash,andskiplistthesparsitycanbecontrolled,fulltextandgeoaresparsebydefinition.
uniquecanbetrueorfalseandissupportedbyhashorskiplist
Callingthismethodreturnsanindexobject.Whetherornottheindexobjectexistedbeforethecallisindicatedinthereturnattribute
isNewlyCreated.
deduplicatecanbetrueorfalseandissupportedbyarrayindexesoftypehashorskiplist.Itcontrolswhetherinsertingduplicateindex
valuesfromthesamedocumentintoauniquearrayindexwillleadtoauniqueconstrainterrorornot.Thedefaultvalueistrue,soonlya
singleinstanceofeachnon-uniqueindexvaluewillbeinsertedintotheindexperdocument.Tryingtoinsertavalueintotheindexthat
alreadyexistsintheindexwillalwaysfail,regardlessofthevalueofthisattribute.
Examples
arangosh>db.test.ensureIndex({type:"hash",fields:["a"],sparse:true});
arangosh>db.test.ensureIndex({type:"hash",fields:["a","b"],unique:true});
showexecutionresults
Droppinganindex
dropsanindex collection.dropIndex(index)
Dropstheindex.Iftheindexdoesnotexist,thenfalseisreturned.Iftheindexexistedandwasdropped,thentrueisreturned.Notethat
youcannotdropsomespecialindexes(e.g.theprimaryindexofacollectionortheedgeindexofanedgecollection).
collection.dropIndex(index-handle)
Sameasabove.Insteadofanindexanindexhandlecanbegiven.
arangosh>db.example.ensureSkiplist("a","b");
arangosh>varindexInfo=db.example.getIndexes();
arangosh>indexInfo;
arangosh>db.example.dropIndex(indexInfo[0])
arangosh>db.example.dropIndex(indexInfo[1].id)
arangosh>indexInfo=db.example.getIndexes();
showexecutionresults
LoadIndexesintoMemory
LoadsallindexesofthiscollectionintoM emory. collection.loadIndexesIntoMemory()
Thisfunctiontriestocacheallindexentriesofthiscollectionintothemainmemory.Thereforeititeratesoverallindexesofthecollection
andstorestheindexedvalues,nottheentiredocumentdata,inmemory.Alllookupsthatcouldbefoundinthecachearemuchfasterthan
lookupsnotstoredinthecachesoyougetaniceperformanceboost.Itisalsoguaranteedthatthecacheisconsistentwiththestored
data.
ForthetimebeingthisfunctionisonlyusefulonRocksDBstorageengine,asinM M Filesengineallindexesareinmemoryanyways.
OnRocksDBthisfunctionhonorsallmemorylimits,iftheindexesyouwanttoloadaresmallerthanyourmemorylimitthisfunction
guaranteesthatmostindexvaluesarecached.Iftheindexislargerthanyourmemorylimitthisfunctionwillfillupvaluesuptothislimit
andforthetimebeingthereisnowaytocontrolwhichindexesofthecollectionshouldhavepriorityoverothers.
arangosh>db.example.loadIndexesIntoMemory();
{
"result":true
}
98
WorkingwithIndexes
DatabaseMethods
Fetchinganindexbyhandle
findsanindex db._index(index-handle)
Returnstheindexwithindex-handleornullifnosuchindexexists.
arangosh>db.example.ensureIndex({type:"skiplist",fields:["a","b"]});
arangosh>varindexInfo=db.example.getIndexes().map(function(x){returnx.id;});
arangosh>indexInfo;
arangosh>db._index(indexInfo[0])
arangosh>db._index(indexInfo[1])
showexecutionresults
Droppinganindex
dropsanindex db._dropIndex(index)
Dropstheindex.Iftheindexdoesnotexist,thenfalseisreturned.Iftheindexexistedandwasdropped,thentrueisreturned.
db._dropIndex(index-handle)
Dropstheindexwithindex-handle.
arangosh>db.example.ensureIndex({type:"skiplist",fields:["a","b"]});
arangosh>varindexInfo=db.example.getIndexes();
arangosh>indexInfo;
arangosh>db._dropIndex(indexInfo[0])
arangosh>db._dropIndex(indexInfo[1].id)
arangosh>indexInfo=db.example.getIndexes();
showexecutionresults
Revalidatingwhetheranindexisused
findsanindex
Soyou'vecreatedanindex,andsinceitsmaintainanceisn'tforfree,youdefinitelywanttoknowwhetheryourquerycanutilizeit.
Youcanuseexplaintoverifywhetherskiplistsorhashindexesareused(ifyouomit colors:falseyouwillgetnicecolorsin
ArangoShell):
arangosh>varexplain=require("@arangodb/aql/explainer").explain;
arangosh>db.example.ensureIndex({type:"skiplist",fields:["a","b"]});
arangosh>explain("FORdocINexampleFILTERdoc.a<23RETURNdoc",{colors:false});
showexecutionresults
99
HashIndexes
HashIndexes
IntroductiontoHashIndexes
Itispossibletodefineahashindexononeormoreattributes(orpaths)ofadocument.Thishashindexisthenusedinqueriestolocate
documentsinO(1)operations.Ifthehashindexisunique,thennotwodocumentsareallowedtohavethesamesetofattributevalues.
Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheindexisdeclaredsparse,adocumentwillbe
excludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueof null.
AccessingHashIndexesfromtheShell
UniqueHashIndexes
Ensuresthatauniqueconstraintexists: collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],unique:true})
Createsauniquehashindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.The
indexwillbenon-sparsebydefault.
Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocument
willwillfailiftheattributeuniquenessisviolated.
Tocreateasparseuniqueindex,setthesparseattributeto true:
collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],unique:true,sparse:true})
Incasethattheindexwassuccessfullycreated,theindexidentifierisreturned.
Non-existingattributeswilldefaultto null.Inasparseindexalldocumentswillbeexcludedfromtheindexforwhichallspecifiedindex
attributesare null.Suchdocumentswillnotbetakenintoaccountforuniquenesschecks.
Inanon-sparseindex,alldocumentsregardlessof null-attributeswillbeindexedandwillbetakenintoaccountforuniqueness
checks.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.test.ensureIndex({type:"hash",fields:["a","b.c"],unique:true});
arangosh>db.test.save({a:1,b:{c:1}});
arangosh>db.test.save({a:1,b:{c:1}});
arangosh>db.test.save({a:1,b:{c:null}});
arangosh>db.test.save({a:1});
showexecutionresults
Non-uniqueHashIndexes
Ensuresthatanon-uniquehashindexexists: collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"]})
Createsanon-uniquehashindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.The
indexwillbenon-sparsebydefault.
Tocreateasparseuniqueindex,setthesparseattributeto true:
collection.ensureIndex({type:"hash",fields:["field1",...,"fieldn"],sparse:true})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.test.ensureIndex({type:"hash",fields:["a"]});
arangosh>db.test.save({a:1});
100
HashIndexes
arangosh>db.test.save({a:1});
arangosh>db.test.save({a:null});
showexecutionresults
HashArrayIndexes
Ensuresthatahasharrayindexexists(non-unique): collection.ensureIndex({type:"hash",fields:["field1[*]",...,"fieldn[*]"]
})
Createsanon-uniquehasharrayindexfortheindividualelementsofthearrayattributesfield1[*],...fieldn[*]foundinthedocuments.At
leastoneattributepathhastobegiven.Theindexalwaystreatstheindexedarraysassparse.
Itispossibletocombinearrayindexingwithstandardindexing: collection.ensureIndex({type:"hash",fields:["field1[*]",
"field2"]})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.test.ensureIndex({type:"hash",fields:["a[*]"]});
arangosh>db.test.save({a:[1,2]});
arangosh>db.test.save({a:[1,3]});
arangosh>db.test.save({a:null});
showexecutionresults
Ensureuniquenessofrelationsinedgecollections
Itispossibletocreatesecondaryindexesusingtheedgeattributes _fromand _to,startingwithArangoDB3.0.Acombinedindexover
bothfieldstogetherwiththeuniqueoptionenabledcanbeusedtopreventduplicaterelationsfrombeingcreated.
Forexample,adocumentcollectionvertsmightcontainverticeswiththedocumenthandles verts/A, verts/Band verts/C.Relations
betweenthesedocumentscanbestoredinanedgecollectionedgesforinstance.Now,youmaywanttomakesurethatthevertex
verts/Aisneverlinkedto verts/Bbyanedgemorethanonce.Thiscanbeachievedbyaddingaunique,non-sparsehashindexforthe
fields _fromand _to:
db.edges.ensureIndex({type:"hash",fields:["_from","_to"],unique:true});
Creatinganedge {_from:"verts/A",_to:"verts/B"}inedgeswillbeaccepted,butonlyonce.Anotherattempttostoreanedgewith
therelationA→Bwillberejectedbytheserverwithauniqueconstraintviolatederror.Thisincludesupdatestothe _fromand _to
fields.
NotethataddingarelationB→Aisstillpossible,soisA→AandB→B,becausetheyarealldifferentrelationsinadirectedgraph.
Eachonecanonlyoccuroncehowever.
101
Skiplists
Skiplists
IntroductiontoSkiplistIndexes
ThisisanintroductiontoArangoDB'sskiplists.
Itispossibletodefineaskiplistindexononeormoreattributes(orpaths)ofdocuments.Thisskiplististhenusedinqueriestolocate
documentswithinagivenrange.Iftheskiplistisdeclaredunique,thennotwodocumentsareallowedtohavethesamesetofattribute
values.
Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheskiplistindexisdeclaredsparse,adocument
willbeexcludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueof null.
AccessingSkiplistIndexesfromtheShell
UniqueSkiplistIndex
Ensuresthatauniqueskiplistindexexists: collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],unique:
true})
Createsauniqueskiplistindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.The
indexwillbenon-sparsebydefault.
Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocument
willfailiftheattributeuniquenessisviolated.
Tocreateasparseuniqueindex,setthesparseattributeto true:
collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],unique:true,sparse:true})
Inasparseindexalldocumentswillbeexcludedfromtheindexthatdonotcontainatleastoneofthespecifiedindexattributesorthat
haveavalueof nullinanyofthespecifiedindexattributes.Suchdocumentswillnotbeindexed,andnotbetakenintoaccountfor
uniquenesschecks.
Inanon-sparseindex,thesedocumentswillbeindexed(fornon-presentindexedattributes,avalueof nullwillbeused)andwillbe
takenintoaccountforuniquenesschecks.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.ids.ensureIndex({type:"skiplist",fields:["myId"],unique:true});
arangosh>db.ids.save({"myId":123});
arangosh>db.ids.save({"myId":456});
arangosh>db.ids.save({"myId":789});
arangosh>db.ids.save({"myId":123});
showexecutionresults
arangosh>db.ids.ensureIndex({type:"skiplist",fields:["name.first","name.last"],
unique:true});
arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});
arangosh>db.ids.save({"name":{"first":"jens","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});
showexecutionresults
Non-uniqueSkiplistIndex
102
Skiplists
Ensuresthatanon-uniqueskiplistindexexists: collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"]})
Createsanon-uniqueskiplistindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.
Theindexwillbenon-sparsebydefault.
Tocreateasparsenon-uniqueindex,setthesparseattributeto true.
collection.ensureIndex({type:"skiplist",fields:["field1",...,"fieldn"],sparse:true})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.names.ensureIndex({type:"skiplist",fields:["first"]});
arangosh>db.names.save({"first":"Tim"});
arangosh>db.names.save({"first":"Tom"});
arangosh>db.names.save({"first":"John"});
arangosh>db.names.save({"first":"Tim"});
arangosh>db.names.save({"first":"Tom"});
showexecutionresults
SkiplistArrayIndex
Ensuresthataskiplistarrayindexexists(non-unique): collection.ensureIndex({type:"skiplist",fields:["field1[*]",...,
"fieldn[*]"]})
Createsanon-uniqueskiplistarrayindexfortheindividualelementsofthearrayattributesfield1[*],...fieldn[*]foundinthedocuments.
Atleastoneattributepathhastobegiven.Theindexalwaystreatstheindexedarraysassparse.
Itispossibletocombinearrayindexingwithstandardindexing: collection.ensureIndex({type:"skiplist",fields:["field1[*]",
"field2"]})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.test.ensureIndex({type:"skiplist",fields:["a[*]"]});
arangosh>db.test.save({a:[1,2]});
arangosh>db.test.save({a:[1,3]});
arangosh>db.test.save({a:null});
showexecutionresults
Querybyexampleusingaskiplistindex
Constructsaquery-by-exampleusingaskiplistindex: collection.byExample(example)
Selectsalldocumentsfromthecollectionthatmatchthespecifiedexampleandreturnsacursor.Askiplistindexwillbeusedifpresent.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.Ifyouuse
{"a":{"c":1}}
asexample,thenyouwillfindalldocuments,suchthattheattributeacontainsadocumentoftheform{c:1}.Forexamplethe
document
{"a":{"c":1},"b":1}
willmatch,butthedocument
{"a":{"c":1,"b":1}}
103
Skiplists
willnot.
However,ifyouuse
{"a.c":1},
thenyouwillfindalldocuments,whichcontainasub-documentinathathasanattributecofvalue1.Boththefollowingdocuments
{"a":{"c":1},"b":1}
and
{"a":{"c":1,"b":1}}
willmatch.
104
Persistent
Persistentindexes
IntroductiontoPersistentIndexes
ThisisanintroductiontoArangoDB'spersistentindexes.
Itispossibletodefineapersistentindexononeormoreattributes(orpaths)ofdocuments.Theindexisthenusedinqueriestolocate
documentswithinagivenrange.Iftheindexisdeclaredunique,thennotwodocumentsareallowedtohavethesamesetofattribute
values.
Creatinganewdocumentorupdatingadocumentwillfailiftheuniquenessisviolated.Iftheindexisdeclaredsparse,adocumentwillbe
excludedfromtheindexandnouniquenesscheckswillbeperformedifanyindexattributevalueisnotsetorhasavalueof null.
AccessingPersistentIndexesfromtheShell
ensuresthatauniquepersistentindexexists collection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"],
unique:true})
Createsauniquepersistentindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobegiven.
Theindexwillbenon-sparsebydefault.
Alldocumentsinthecollectionmustdifferintermsoftheindexedattributes.Creatinganewdocumentorupdatinganexistingdocument
willwillfailiftheattributeuniquenessisviolated.
Tocreateasparseuniqueindex,setthesparseattributeto true:
collection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"],unique:true,sparse:true})
Inasparseindexalldocumentswillbeexcludedfromtheindexthatdonotcontainatleastoneofthespecifiedindexattributesorthat
haveavalueof nullinanyofthespecifiedindexattributes.Suchdocumentswillnotbeindexed,andnotbetakenintoaccountfor
uniquenesschecks.
Inanon-sparseindex,thesedocumentswillbeindexed(fornon-presentindexedattributes,avalueof nullwillbeused)andwillbe
takenintoaccountforuniquenesschecks.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.ids.ensureIndex({type:"persistent",fields:["myId"],unique:true});
arangosh>db.ids.save({"myId":123});
arangosh>db.ids.save({"myId":456});
arangosh>db.ids.save({"myId":789});
arangosh>db.ids.save({"myId":123});
showexecutionresults
arangosh>db.ids.ensureIndex({type:"persistent",fields:["name.first","name.last"],
unique:true});
arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});
arangosh>db.ids.save({"name":{"first":"jens","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"jensen"}});
arangosh>db.ids.save({"name":{"first":"hans","last":"hansen"}});
showexecutionresults
ensuresthatanon-uniquepersistentindexexists collection.ensureIndex({type:"persistent",fields:["field1",...,"fieldn"]})
Createsanon-uniquepersistentindexonalldocumentsusingfield1,...fieldnasattributepaths.Atleastoneattributepathhastobe
given.Theindexwillbenon-sparsebydefault.
105
Persistent
Tocreateasparseuniqueindex,setthesparseattributeto true.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
arangosh>db.names.ensureIndex({type:"persistent",fields:["first"]});
arangosh>db.names.save({"first":"Tim"});
arangosh>db.names.save({"first":"Tom"});
arangosh>db.names.save({"first":"John"});
arangosh>db.names.save({"first":"Tim"});
arangosh>db.names.save({"first":"Tom"});
showexecutionresults
Querybyexampleusingapersistentindex
constructsaquery-by-exampleusingapersistentindex collection.byExample(example)
Selectsalldocumentsfromthecollectionthatmatchthespecifiedexampleandreturnsacursor.Apersistentindexwillbeusedifpresent.
YoucanusetoArray,next,orhasNexttoaccesstheresult.Theresultcanbelimitedusingtheskipandlimitoperator.
Anattributenameoftheforma.bisinterpretedasattributepath,notasattribute.Ifyouuse
{"a":{"c":1}}
asexample,thenyouwillfindalldocuments,suchthattheattributeacontainsadocumentoftheform{c:1}.Forexamplethe
document
{"a":{"c":1},"b":1}
willmatch,butthedocument
{"a":{"c":1,"b":1}}
willnot.
However,ifyouuse
{"a.c":1},
thenyouwillfindalldocuments,whichcontainasub-documentinathathasanattributecofvalue1.Boththefollowingdocuments
{"a":{"c":1},"b":1}
and
{"a":{"c":1,"b":1}}
willmatch.
PersistentIndexesandServerLanguage
Theorderofindexentriesinpersistentindexesadherestotheconfiguredserverlanguage.If,however,theserverisrestartedwitha
differentlanguagesettingaswhenthepersistentindexwascreated,notalldocumentsmaybereturnedanymoreandthesortorderof
thosewhicharereturnedcanbewrong(wheneverthepersistentindexisconsulted).
Tofixpersistentindexesafteralanguagechange,deleteandre-createthem.Skiplistindexesarenotaffected,becausetheyarenot
persistedandautomaticallyrebuiltoneveryserverstart.
106
Persistent
107
FulltextIndexes
Fulltextindexes
ThisisanintroductiontoArangoDB'sfulltextindexes.
IntroductiontoFulltextIndexes
Afulltextindexcanbeusedtofindwords,orprefixesofwordsinsidedocuments.
Afulltextindexcanbedefinedononeattributeonly,andwillincludeallwordscontainedindocumentsthathaveatextualvalueinthe
indexattribute.SinceArangoDB2.6theindexwillalsoincludewordsfromtheindexattributeiftheindexattributeisanarrayofstrings,
oranobjectwithstringvaluemembers.
Forexample,givenafulltextindexonthe translationsattributeandthefollowingdocuments,thensearchingfor лисаusingthe
fulltextindexwouldreturnonlythefirstdocument.Searchingfortheindexfortheexactstring Foxwouldreturnthefirsttwo
documents,andsearchingfor prefix:Foxwouldreturnallthreedocuments:
{translations:{en:"fox",de:"Fuchs",fr:"renard",ru:"лиса"}}
{translations:"FoxistheEnglishtranslationoftheGermanwordFuchs"}
{translations:["ArangoDB","document","database","Foxx"]}
Notethatdeepernestedobjectsareignored.Forexample,afulltextindexontranslationswouldindexFuchs,butnotfox,giventhe
followingdocumentstructure:
{translations:{en:{US:"fox"},de:"Fuchs"}
Ifyouneedtosearchacrossmultiplefieldsand/ornestedobjects,youmaywriteallthestringsintoaspecialattribute,whichyouthen
createtheindexon(itmightbenecessarytocleanthestringsfirst,e.g.removelinebreaksandstripcertainwords).
Iftheindexattributeisneitherastring,anobjectoranarray,itscontentswillnotbeindexed.Whenindexingthecontentsofanarray
attribute,anarraymemberwillonlybeincludedintheindexifitisastring.Whenindexingthecontentsofanobjectattribute,anobject
membervaluewillonlybeincludedintheindexifitisastring.Otherdatatypesareignoredandnotindexed.
Currently,fulltextindexesarenotyetsupportedwiththeRocksDBstorageengine.Thusthefunction FULLTEXT()willbeunavailable
whenusingthisstorageengine.Tousefulltextindexes,pleaseusetheM M Filesstorageengineforthetimebeing.
AccessingFulltextIndexesfromtheShell
Ensuresthatafulltextindexexists:
collection.ensureIndex({type:"fulltext",fields:["field"],minLength:minLength})
Createsafulltextindexonalldocumentsonattributefield.
Fulltextindexesareimplicitlysparse:alldocumentswhichdonothavethespecifiedfieldattributeorthathaveanon-qualifyingvaluein
theirfieldattributewillbeignoredforindexing.
Onlyasingleattributecanbeindexed.Specifyingmultipleattributesisunsupported.
TheminimumlengthofwordsthatareindexedcanbespecifiedviatheminLengthparameter.WordsshorterthanminLengthcharacters
willnotbeindexed.minLengthhasadefaultvalueof2,butthisvaluemightbechangedinfutureversionsofArangoDB.Itisthus
recommendedtoexplicitlyspecifythisvalue.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetailsisreturned.
arangosh>db.example.ensureIndex({type:"fulltext",fields:["text"],minLength:3});
arangosh>db.example.save({text:"thequickbrown",b:{c:1}});
arangosh>db.example.save({text:"quickbrownfox",b:{c:2}});
arangosh>db.example.save({text:"brownfoxjums",b:{c:3}});
arangosh>db.example.save({text:"foxjumpsover",b:{c:4}});
108
FulltextIndexes
arangosh>db.example.save({text:"jumpsoverthe",b:{c:5}});
arangosh>db.example.save({text:"overthelazy",b:{c:6}});
arangosh>db.example.save({text:"thelazydog",b:{c:7}});
arangosh>db._query("FORdocumentINFULLTEXT(example,'text','the')RETURNdocument");
showexecutionresults
Looksupafulltextindex:
collection.lookupFulltextIndex(attribute,minLength)
Checkswhetherafulltextindexonthegivenattributeattributeexists.
FulltextAQLFunctions
FulltextAQLfunctionsaredetailedinFulltextfunctions.
109
GeoIndexes
GeoIndexes
IntroductiontoGeoIndexes
ThisisanintroductiontoArangoDB'sgeoindexes.
AQL'sgeographicfeaturesaredescribedinGeofunctions.
ArangoDBusesHilbertcurvestoimplementgeo-spatialindexes.Seethisblogfordetails.
Ageo-spatialindexassumesthatthelatitudeisbetween-90and90degreeandthelongitudeisbetween-180and180degree.Ageoindex
willignorealldocumentswhichdonotfulfilltheserequirements.
AccessingGeoIndexesfromtheShell
ensuresthatageoindexexists collection.ensureIndex({type:"geo",fields:["location"]})
Createsageo-spatialindexonalldocumentsusinglocationaspathtothecoordinates.Thevalueoftheattributehastobeanarraywithat
leasttwonumericvalues.Thearraymustcontainthelatitude(firstvalue)andthelongitude(secondvalue).
Alldocuments,whichdonothavetheattributepathorhaveanon-conformingvalueinitareexcludedfromtheindex.
Ageoindexisimplicitlysparse,andthereisnowaytocontrolitssparsity.
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
Tocreateageoindexonanarrayattributethatcontainslongitudefirst,setthegeoJsonattributeto true.Thiscorrespondstothe
formatdescribedinRFC7946Position
collection.ensureIndex({type:"geo",fields:["location"],geoJson:true})
Tocreateageo-spatialindexonalldocumentsusinglatitudeandlongitudeasseparateattributepaths,twopathsneedtobespecifiedin
thefieldsarray:
collection.ensureIndex({type:"geo",fields:["latitude","longitude"]})
Incasethattheindexwassuccessfullycreated,anobjectwiththeindexdetails,includingtheindex-identifier,isreturned.
Examples
Createageoindexforanarrayattribute:
arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});
........>}
........>}
arangosh>db.geo.count();
arangosh>db.geo.near(0,0).limit(3).toArray();
arangosh>db.geo.near(0,0).count();
showexecutionresults
Createageoindexforahasharrayattribute:
arangosh>db.geo2.ensureIndex({type:"geo",fields:["location.latitude",
"location.longitude"]});
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.geo2.save({name:"Name/"+i+"/"+j,location:{latitude:i,
longitude:j}});
110
GeoIndexes
........>}
........>}
arangosh>db.geo2.near(0,0).limit(3).toArray();
showexecutionresults
UseGeoIndexwithAQLSORTstatement:
arangosh>db.geoSort.ensureIndex({type:"geo",fields:["latitude","longitude"]});
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.geoSort.save({name:"Name/"+i+"/"+j,latitude:i,longitude
:j});
........>}
........>}
arangosh>varquery="FORdocingeoSortSORTDISTANCE(doc.latitude,doc.longitude,0,0)
LIMIT5RETURNdoc"
arangosh>db._explain(query,{},{colors:false});
arangosh>db._query(query);
showexecutionresults
UseGeoIndexwithAQLFILTERstatement:
arangosh>db.geoFilter.ensureIndex({type:"geo",fields:["latitude","longitude"]});
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.geoFilter.save({name:"Name/"+i+"/"+j,latitude:i,
longitude:j});
........>}
........>}
arangosh>varquery="FORdocingeoFilterFILTERDISTANCE(doc.latitude,doc.longitude,
0,0)<2000RETURNdoc"
arangosh>db._explain(query,{},{colors:false});
arangosh>db._query(query);
showexecutionresults
constructsageoindexselection collection.geo(location-attribute)Looksupageoindexdefinedonattributelocation_attribute.
Returnsageoindexobjectifanindexwasfound.The nearor withinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthis
particularindex.Thisisusefulforcollectionswithmultipledefinedgeoindexes. collection.geo(location_attribute,true)Looksupa
geoindexonacompoundattributelocation_attribute.Returnsageoindexobjectifanindexwasfound.The nearor withinoperators
canthenbeusedtoexecuteageo-spatialqueryonthisparticularindex. collection.geo(latitude_attribute,longitude_attribute)
Looksupageoindexdefinedonthetwoattributeslatitude_attributeandlongitude-attribute.Returnsageoindexobjectifanindexwas
found.The nearor withinoperatorscanthenbeusedtoexecuteageo-spatialqueryonthisparticularindex.Note:thismethodisnot
yetsupportedbytheRocksDBstorageengine.Note:thegeosimplequeryhelperfunctionisdeprecatedasofArangoDB2.6.The
functionmayberemovedinfutureversionsofArangoDB.ThepreferredwayforrunninggeoqueriesistousetheirAQLequivalents.
Examples
Assumeyouhavealocationstoredaslistintheattributehomeandadestinationstoredintheattributework.Thenyoucanusethe geo
operatortoselectwhichgeo-spatialattributes(andthuswhichindex)touseina nearquery.
arangosh>for(i=-90;i<=90;i+=10){
........>for(j=-180;j<=180;j+=10){
........>db.complex.save({name:"Name/"+i+"/"+j,
........>home:[i,j],
........>work:[-i,-j]});
........>}
111
GeoIndexes
........>}
........>
arangosh>db.complex.near(0,170).limit(5);
arangosh>db.complex.ensureIndex({type:"geo",fields:["home"]});
arangosh>db.complex.near(0,170).limit(5).toArray();
arangosh>db.complex.geo("work").near(0,170).limit(5);
arangosh>db.complex.ensureIndex({type:"geo",fields:["work"]});
arangosh>db.complex.geo("work").near(0,170).limit(5).toArray();
showexecutionresults
constructsanearqueryforacollection collection.near(latitude,longitude)Thereturnedlistissortedaccordingtothedistance,with
thenearestdocumenttothecoordinate(latitude,longitude)comingfirst.Ifthereareneardocumentsofequaldistance,documentsare
chosenrandomlyfromthissetuntilthelimitisreached.Itispossibletochangethelimitusingthelimitoperator.Inordertousethenear
operator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesforthedocument.If
youhavemorethenonegeo-spatialindex,youcanusethegeooperatortoselectaparticularindex.Note: neardoesnotsupport
negativeskips.//However,youcanstilluse limitfollowedtoskip. collection.near(latitude,longitude).limit(limit)Limitsthe
resulttolimitdocumentsinsteadofthedefault100.Note:Unlikewithmultipleexplicitlimits, limitwillraisetheimplicitdefaultlimit
imposedby within. collection.near(latitude,longitude).distance()Thiswilladdanattribute distancetoalldocuments
returned,whichcontainsthedistancebetweenthegivenpointandthedocumentinmeters. collection.near(latitude,
longitude).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegiven
pointandthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thenearsimplequery
functionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.Thepreferredwayfor
retrievingdocumentsfromacollectionusingthenearoperatoristousetheAQLNEARfunctioninanAQLqueryasfollows:
FORdocINNEAR(@@collection,@latitude,@longitude,@limit)
RETURNdoc
Examples
Togetthenearesttwolocations:
arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});
arangosh>for(vari=-90;i<=90;i+=10){
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({
........>name:"Name/"+i+"/"+j,
........>loc:[i,j]});
........>}}
arangosh>db.geo.near(0,0).limit(2).toArray();
showexecutionresults
Ifyouneedthedistanceaswell,thenyoucanusethe distanceoperator:
arangosh>db.geo.ensureIndex({type:"geo",fields:["loc"]});
arangosh>for(vari=-90;i<=90;i+=10){
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({
........>name:"Name/"+i+"/"+j,
........>loc:[i,j]});
........>}}
arangosh>db.geo.near(0,0).distance().limit(2).toArray();
showexecutionresults
constructsawithinqueryforacollection collection.within(latitude,longitude,radius)Thiswillfindalldocumentswithinagiven
radiusaroundthecoordinate(latitude,longitude).Thereturnedarrayissortedbydistance,beginningwiththenearestdocument.Inorder
tousethewithinoperator,ageoindexmustbedefinedforthecollection.Thisindexalsodefineswhichattributeholdsthecoordinatesfor
112
GeoIndexes
thedocument.Ifyouhavemorethenonegeo-spatialindex,youcanusethe geooperatortoselectaparticularindex.
collection.within(latitude,longitude,radius).distance()Thiswilladdanattribute _distancetoalldocumentsreturned,which
containsthedistancebetweenthegivenpointandthedocumentinmeters. collection.within(latitude,longitude,
radius).distance(name)Thiswilladdanattributenametoalldocumentsreturned,whichcontainsthedistancebetweenthegivenpoint
andthedocumentinmeters.Note:thismethodisnotyetsupportedbytheRocksDBstorageengine.Note:thewithinsimplequery
functionisdeprecatedasofArangoDB2.6.ThefunctionmayberemovedinfutureversionsofArangoDB.Thepreferredwayfor
retrievingdocumentsfromacollectionusingthewithinoperatoristousetheAQLWITHINfunctioninanAQLqueryasfollows:
FORdocINWITHIN(@@collection,@latitude,@longitude,@radius,@distanceAttributeName)
RETURNdoc
Examples
Tofindalldocumentswithinaradiusof2000kmuse:
arangosh>for(vari=-90;i<=90;i+=10){
........>for(varj=-180;j<=180;j+=10){
........>db.geo.save({name:"Name/"+i+"/"+j,loc:[i,j]});}}
arangosh>db.geo.within(0,0,2000*1000).distance().toArray();
showexecutionresults
ensuresthatageoindexexists collection.ensureIndex({type:"geo",fields:["location"]})
SinceArangoDB2.5,thismethodisanaliasforensureGeoIndexsincegeoindexesarealwayssparse,meaningthatdocumentsthatdonot
containtheindexattributesorhavenon-numericvaluesintheindexattributeswillnotbeindexed.ensureGeoConstraintisdeprecatedand
ensureGeoIndexshouldbeusedinstead.
Theindexdoesnotprovidea uniqueoptionbecauseofitslimitedusability.Itwouldpreventidenticalcoordinatesfrombeinginserted
only,butevenaslightlydifferentlocation(like1inchor1cmoff)wouldbeuniqueagainandnotconsideredaduplicate,althoughit
probablyshould.Thedesiredthresholdfordetectingduplicatesmayvaryforeveryproject(includinghowtocalculatethedistanceeven)
andneedstobeimplementedontheapplicationlayerasneeded.YoucanwriteaFoxxserviceforthispurposeandmakeuseoftheAQL
geofunctionstofindnearbycoordinatessupportedbyageoindex.
113
VertexCentricIndexes
VertexCentricIndexes
IntroductiontoVertexCentricIndexes
InArangoDBtherearespecialindicesdesignedtospeedupgraphoperations,especiallyifthegraphcontainssupernodes(verticesthat
haveanexceptionallyhighamountofconnectededges).Theseindicesarecalledvertexcentricindexesandcanbeusedinadditiontothe
existingedgeindex.
Motivation
Theideaofthisindexistoindexacombinationofavertex,thedirectionandanyarbitrarysetofotherattributesontheedges.Totakean
example,ifwehaveanattributecalled typeontheedges,wecanuseanoutboundvertex-centricindexonthisattributetofindalledges
attachedtoavertexwithagiven type.Thefollowingqueryexamplecouldbenefitfromsuchanindex:
FORv,e,pIN3..5OUTBOUND@startGRAPH@graphName
FILTERp.edges[*].typeALL=="friend"
RETURNv
Usingthebuilt-inedge-indexArangoDBcanfindthelistofalledgesattachedtothevertexfast,butstillithastowalkthroughthislist
andcheckifallofthemhavetheattribute type=="friend".Usingavertex-centricindexwouldallowArangoDBtofindalledgesfor
thevertexhavingtheattribute type=="friend"inthesametimeandcansavetheiterationtoverifythecondition.
Indexcreation
Avertex-centriccanbeeitherofthefollowingtypes:
HashIndex
SkiplistIndex
PersistentIndex
Andiscreatedusingtheircreationoperations.Howeverinthelistoffieldsusedtocreatetheindexwehavetoincludeeither _fromor
_to.Letusagainexplainthisbyanexample.Assumewewanttocreateanhash-basedoutboundvertex-centricindexontheattribute
type.Thiscanbecreatedwiththefollowingway:
arangosh>db.collection.ensureIndex({type:"hash",fields:["_from","type"]})
showexecutionresults
Alloptionsthataresupportedbytherespectiveindexesaresupportedbythevertex-centricindexaswell.
Indexusage
TheAQLoptimizercandecidetouseavertex-centricwheneversuitable,howeveritisnotguaranteedthatthisindexisused,the
optimizermayestimatethatanotherindexisassumedtobebetter.Theoptimizerwillconsiderthistypeofindexesonexplicitfiltering
of _fromrespectively _to:
FORedgeINcollection
FILTERedge._from=="vertices/123456"ANDedge.type=="friend"
RETURNedge
andduringpatternmatchingqueries:
FORv,e,pIN3..5OUTBOUND@startGRAPH@graphName
FILTERp.edges[*].typeALL=="friend"
RETURNv
114
VertexCentricIndexes
115
Graphs
ArangoDBGraphs
FirstStepswithGraphs
AGraphconsistsofverticesandedges.Edgesarestoredasdocumentsinedgecollections.Avertexcanbeadocumentofadocument
collectionorofanedgecollection(soedgescanbeusedasvertices).Whichcollectionsareusedwithinanamedgraphisdefinedviaedge
definitions.Anamedgraphcancontainmorethanoneedgedefinition,atleastoneisneeded.Graphsallowyoutostructureyourmodels
inlinewithyourdomainandgroupthemlogicallyincollectionsandgivingyouthepowertoquerytheminthesamegraphqueries.
Newtographs?Takeourfreegraphcourseforfreshersandgetfromzeroknowledgetoadvancedquerytechniques.
Comingfromarelationalbackground-what'sagraph?
InSQLyoucommonlyhavetheconstructofarelationtabletostoren:mrelationsbetweentwodatatables.Anedgecollectionis
somewhatsimilartotheserelationtables;vertexcollectionsresemblethedatatableswiththeobjectstoconnect.Whilesimplegraph
querieswithfixednumberofhopsviatherelationtablemaybedoableinSQLwithseveralnestedjoins,graphdatabasescanhandlean
arbitrarynumberofthesehopsoveredgecollections-thisiscalledtraversal.Alsoedgesinoneedgecollectionmaypointtoseveral
vertexcollections.Itscommontohaveattributesattachedtoedges,i.e.alabelnamingthisinterconnection.Edgeshaveadirection,with
theirrelations _fromand _topointingfromonedocumenttoanotherdocumentstoredinvertexcollections.Inqueriesyoucandefine
inwhichdirectionstheedgerelationsmaybefollowed( OUTBOUND: _from→ _to, INBOUND: _from← _to, ANY: _from↔
_to).
NamedGraphs
Namedgraphsarecompletelymanagedbyarangodb,andthusalsovisibleinthewebinterface.TheyusethefullspectrumofArangoDBs
graphfeatures.Youmayaccessthemviaseveralinterfaces.
AQLGraphOperationswithseveralflavors:
AQLTraversalsonbothnamedandanonymousgraphs
AQLShortestPathonbothnamedandanonymousgraph
JavaScriptGeneralGraphimplementation,asyoumayuseitinFoxxServices
GraphM anagement;creating&manipualatinggraphdefinitions;inserting,updatinganddeletingverticesandedgesintographs
GraphFunctionsforworkingwithedgesandvertices,toanalyzethemandtheirrelations
JavaScriptSmartGraphimplementation,forscalablegraphs
SmartGraphM anagement;creating&manipualatingSmartGraphdefinitions;DifferencestoGeneralGraph
RESTfulGeneralGraphinterfaceusedtoimplementgraphmanagementinclientdrivers
Manipulatingcollectionsofnamedgraphswithregulardocumentfunctions
Theunderlyingcollectionsofthenamedgraphsarestillaccessibleusingthestandardmethodsforcollections.Howeverthegraphmodule
addsanadditionallayerontopofthesecollectionsgivingyouthefollowingguarantees:
Allmodificationsareexecutedtransactional
Ifyoudeleteavertexalledgeswillbedeleted,youwillneverhavelooseends
Ifyouinsertanedgeitischeckediftheedgematchestheedgedefinitions,youredgecollectionswillonlycontainvalidedges
TheseguaranteesarelostifyouaccessthecollectionsinanyotherwaythanthegraphmoduleorAQL,soifyoudeletedocumentsfrom
yourvertexcollectionsdirectly,theedgespointingtothemwillberemaininplace.
Anonymousgraphs
Sometimesyoumaynotneedallthepowersofnamedgraphs,butsomeofitsbitsmaybevaluabletoyou.Youmayuseanonymous
graphsinthetraversalsandintheWorkingwithEdgeschapter.Anonymousgraphsdon'thaveedgedefinitionsdescribingwhichvertex
collectionisconnectedbywhichedgecollection.Thegraphmodelhastobemaintainedintheclientsidecode.Thisgivesyoumore
freedomthanthestrictnamedgraphs.
116
Graphs
AQLGraphOperationsareavailableforboth,namedandanonymousgraphs:
AQLTraversals
AQLShortestPath
Whentochooseanonymousornamedgraphs?
Asnotedabove,namedgraphsensuregraphintegrity,bothwheninsertingorremovingedgesorvertices.Soyouwon'tencounterdangling
edges,evenifyouusethesamevertexcollectioninseveralnamedgraphs.Thisinvolvesmoreoperationsinsidethedatabasewhichcome
atacost.Thereforeanonymousgraphsmaybefasterinmanyoperations.Sothisquestionmaybenarroweddownto:'CanIaffordthe
additionaleffortordoIneedthewarrantyforintegrity?'.
Multipleedgecollectionsvs. FILTERsonedgedocumentattributes
Ifyouwanttoonlytraverseedgesofaspecifictype,therearetwowaystoachievethis.Thefirstwouldbeanattributeintheedge
document-i.e. type,whereyouspecifyadifferentiatorfortheedge-i.e. "friends", "family", "married"or "workmates",soyou
canlater FILTERe.type="friends"ifyouonlywanttofollowthefriendedges.
Anotherway,whichmaybemoreefficientinsomecases,istousedifferentedgecollectionsfordifferenttypesofedges,soyouhave
friend_eges, family_edges, married_edgesand workmate_edgesascollectionnames.Youcanthenconfigureseveralnamedgraphs
includingasubsetoftheavailableedgeandvertexcollections-oryouuseanonymousgraphqueries,whereyouspecifyalistofedge
collectionstotakeintoaccountinthatquery.Toonlyfollowfriendedges,youwouldspecify friend_edgesassoleedgecollection.
Bothapproacheshaveadvantagesanddisadvantages. FILTERoperationsonedeattributeswilldocomparisonsoneachtraversededge,
whichmaybecomeCPU-intense.Whennotfindingtheedgesinthefirstplacebecauseofthecollectioncontainingthemisnottraversed
atall,therewillneverbeareasontoactualycheckfortheir typeattributewith FILTER.
Themultipleedgecollectionsapproachislimitedbythenumberofcollectionsthatcanbeusedsimultaneouslyinonequery.Every
collectionusedinaqueryrequiressomeresourcesinsideofArangoDBandthenumberisthereforelimitedtocaptheresource
requirements.Youmayalsohaveconstraintsonotheredgeattributes,suchasahashindexwithauniqueconstraint,whichrequiresthe
documentstobeinasinglecollectionfortheuniquenessguarantee,anditmaythusnotbepossibletostorethedifferenttypesofedges
inmultipleedegcollections.
So,ifyouredgeshaveaboutadozendifferenttypes,it'sokaytochoosethecollectionapproach,otherwisethe FILTERapproachis
preferred.Youcanstilluse FILTERoperationsonedgesofcourse.Youcangetridofa FILTERonthe typewiththeformerapproach,
everythingelsecanstaythesame.
WhichpartofmydataisanEdgeandwhichaVertex?
Themainobjectsinyourdatamodel,suchasusers,groupsorarticles,areusuallyconsideredtobevertices.Foreachtypeofobject,a
documentcollection(alsocalledvertexcollection)shouldstoretheindividualentities.Entitiescanbeconnectedbyedgestoexpressand
classifyrelationsbetweenvertices.Itoftenmakessensetohaveanedgecollectionperrelationtype.
ArangoDBdoesnotrequireyoutostoreyourdataingraphstructureswithedgesandvertices,youcanalsodecidetoembedattributes
suchaswhichgroupsauserispartof,or _idsofdocumentsinanotherdocumentinsteadofconnectingthedocumentswithedges.It
canbeameaningfulperformanceoptimizationfor1:nrelationships,ifyourdataisnotfocusedonrelationsandyoudon'tneedgraph
traversalwithvaryingdepth.Itusuallymeanstointroducesomeredundancyandpossiblyinconsistenciesifyouembeddata,butitcan
beanacceptabletradeoff.
Vertices
Let'ssaywehavetwovertexcollections, Usersand Groups.Documentsinthe GroupscollectioncontaintheattributesoftheGroup,
i.e.whenitwasfounded,itssubject,aniconURLandsoon. Usersdocumentscontainthedataspecifictoauser-likeallnames,
birthdays,AvatarURLs,hobbies...
Edges
Wecanuseanedgecollectiontostorerelationsbetweenusersandgroups.Sincemultipleusersmaybeinanarbitrarynumberofgroups,
thisisanm:nrelation.Theedgecollectioncanbecalled UsersInGroupswithi.e.oneedgewith _frompointingto Users/Johnand
_topointingto Groups/BowlingGroupHappyPin.ThismakestheuserJohnamemberofthegroupBowlingGroupHappyPin.
117
Graphs
Attributesofthisrelationmaycontainqualifierstothisrelation,likethepermissionsofJohninthisgroup,thedatewhenhejoinedthe
groupetc.
Soroughlyput,ifyouusedocumentsandtheirattributesinasentence,nounswouldtypicallybevertices,verbsbecometheedges.You
canseethisintheknowsgraphbelow:
AliceknowsBob,whointermknowsCharlie.
Advantagesofthisapproach
Graphsgiveyoutheadvantageofnotjustbeingabletohaveafixednumberofm:nrelationsinarow,butanarbitrarynumber.Edgescan
betraversedinbothdirections,soit'seasytodetermineallgroupsauserisin,butalsotofindoutwhichmembersacertaingrouphas.
Userscouldalsobeinterconnectedtocreateasocialnetwork.
Usingthegraphdatamodel,dealingwithdatathathaslotsofrelationsstaysmanageableandcanbequeriedinveryflexibleways,
whereasitwouldcauseheadachetohandleitinarelationaldatabasesystem.
Backupandrestore
Forsureyouwanttohavebackupsofyourgraphdata,youcanuseArangodumptocreatethebackup,andArangorestoretorestorea
backupintoanewArangoDB.Youshouldhowevernotethat:
youneedthesystemcollection _graphsifyoubackupnamedgraphs.
youneedtobackupthecompletesetofalledgeandvertexcollectionsyourgraphconsistsof.Partialdump/restoremaynotwork.
ExampleGraphs
ArangoDBcomeswithasetofeasilygraspablegraphsthatareusedtodemonstratetheAPIs.Youcanusethe addsamplestabinthe
creategraphwindowinthewebinterface,orloadthemodule @arangodb/graph-examples/example-graphinarangoshanduseittocreate
instancesofthesegraphsinyourArangoDB.Onceyou'vecreatedthem,youcaninspecttheminthewebinterface-whichwasusedto
createthepicturesbelow.
Youcaneasilylookintotheinnardsofthisscriptforreferenceabouthowtomanagegraphsprogramatically.
TheKnows_Graph
118
Graphs
Asetofpersonsknowingeachother:
Theknowsgraphconsistsofonevertexcollection personsconnectedviaoneedgecollection knows.ItwillcontainfivepersonsAlice,
Bob,Charlie,DaveandEve.Wewillhavethefollowingdirectedrelations:
AliceknowsBob
BobknowsCharlie
BobknowsDave
EveknowsAlice
EveknowsBob
Thisishowwecreateit,inspectitsverticesandedges,anddropitagain:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("knows_graph");
arangosh>db.persons.toArray()
arangosh>db.knows.toArray();
arangosh>examples.dropGraph("knows_graph");
showexecutionresults
TheSocialGraph
Asetofpersonsandtheirrelations:
119
Graphs
Thisexamplehasfemaleandmalepersonsasverticesintwovertexcollections- femaleand male.Theedgesaretheirconnectionsin
the relationedgecollection.Thisishowwecreateit,inspectitsverticesandedges,anddropitagain:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>db.female.toArray()
arangosh>db.male.toArray()
arangosh>db.relation.toArray()
arangosh>examples.dropGraph("social");
showexecutionresults
TheCityGraph
Asetofeuropeancities,andtheirfictionaltravelingdistancesasconnections:
120
Graphs
Theexamplehasthecitiesasverticesinseveralvertexcollections- germanCityand frenchCity.Theedgesaretheirinterconnectionsin
severaledgecollections french/german/internationalHighway.Thisishowwecreateit,inspectitsedgesandvertices,anddropit
again:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>db.frenchCity.toArray();
arangosh>db.germanCity.toArray();
arangosh>db.germanHighway.toArray();
arangosh>db.frenchHighway.toArray();
arangosh>db.internationalHighway.toArray();
arangosh>examples.dropGraph("routeplanner");
showexecutionresults
TheTraversalGraph
Thisgraphwasdesignedtodemonstratefiltersintraversals.Ithassomelabelstofilteronit.
121
Graphs
Theexamplehasallitsverticesinthecirclescollection,andanedgesedgecollectiontoconnectthem.Circleshaveuniquenumericlabels.
Edgeshavetwobooleanattributes(theFalsealwaysbeingfalse,theTruthalwaysbeingtrue)andalabelsortingB-Dtotheleftside,GKtotherightside.LeftandrightsidesplitintoPaths-atBandGwhichareeachdirectneighboursoftheroot-nodeA.StartingfromA
thegraphhasadepthof3onallitspaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("traversalGraph");
arangosh>db.circles.toArray();
arangosh>db.edges.toArray();
arangosh>examples.dropGraph("traversalGraph");
showexecutionresults
TheWorldGraph
Theworldcountrygraphstructuresitsnodeslikethat:world→continent→country→capital.Insomecasesedgedirectionsaren't
forward(thereforeitwillbedisplayeddisjunctinthegraphviewer).Ithastwowaysofcreatingit.Oneusingthenamedgraphutilities
(worldCountry),onewithout(worldCountryUnManaged).Itisusedtodemonstraterawtraversaloperations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("worldCountry");
arangosh>db.worldVertices.toArray();
arangosh>db.worldEdges.toArray();
arangosh>examples.dropGraph("worldCountry");
arangosh>varg=examples.loadGraph("worldCountryUnManaged");
arangosh>examples.dropGraph("worldCountryUnManaged");
showexecutionresults
122
Graphs
Cookbookexamples
TheabovereferencedchaptersdescribethevariousAPIsofArangoDBsgraphenginewithsmallexamples.Ourcookbookhassomemore
reallifeexamples:
Traversingagraphinfulldepth
Usinganexamplevertexwiththejavadriver
RetrievingdocumentsfromArangoDBwithoutknowingthestructure
Usingacustomvisitorfromnode.js
AQLExampleQueriesonanActorsandM oviesDatabase
Highervolumegraphexamples
Alloftheaboveexamplesarerathersmallsotheyareeasiertocomprehendandcandemonstratethewaythefunctionalityworks.There
arehoweverseveraldatasetsfreelyavailableonthewebthatarealotbigger.Wecollectedsomeofthemwithimportscriptssoyoumay
playaroundwiththem.AnotherhugegraphisthePokecsocialnetworkfromSlovakiathatweusedforperformancetestingonseveral
databases;Youwillfindimportingscriptsetc.inthisblogpost.
123
GeneralGraphs
Graphs
Thischapterdescribesthegeneral-graphmodule.Itallowsyoutodefineagraphthatisspreadacrossseveraledgeanddocument
collections.Thisallowsyoutostructureyourmodelsinlinewithyourdomainandgroupthemlogicallyincollectionsgivingyouthe
powertoquerytheminthesamegraphqueries.Thereisnoneedtoincludethereferencedcollectionswithinthequery,thismodulewill
handleitforyou.
ThreeStepstocreateagraph
Createagraph
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>vargraph=graph_module._create("myGraph");
arangosh>graph;
{[Graph]
}
Addsomevertexcollections
arangosh>graph._addVertexCollection("shop");
arangosh>graph._addVertexCollection("customer");
arangosh>graph._addVertexCollection("pet");
arangosh>graph;
showexecutionresults
DefinerelationsontheGraph
arangosh>varrel=graph_module._relation("isCustomer",["shop"],["customer"]);
arangosh>graph._extendEdgeDefinitions(rel);
arangosh>graph;
showexecutionresults
124
GraphM anagement
GraphManagement
Thischapterdescribesthejavascriptinterfaceforcreatingandmodifyingnamedgraphs.Inordertocreateanonemptygraphthe
functionalitytocreateedgedefinitionshastobeintroducedfirst:
EdgeDefinitions
Anedgedefinitionisalwaysadirectedrelationofagraph.Eachgraphcanhavearbitrarymanyrelationsdefinedwithintheedge
definitionsarray.
Initializethelist
Createalistofedgedefinitionstoconstructagraph.
graph_module._edgeDefinitions(relation1,relation2,...,relationN)
Thelistofedgedefinitionsofagraphcanbemanagedbythegraphmoduleitself.Thisfunctionistheentrypointforthemanagementand
willreturnthecorrectlist.
Parameters
relationX(optional)Anobjectrepresentingadefinitionofonerelationinthegraph
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>directed_relation=graph_module._relation("lives_in","user","city");
arangosh>undirected_relation=graph_module._relation("knows","user","user");
arangosh>edgedefinitions=graph_module._edgeDefinitions(directed_relation,
undirected_relation);
showexecutionresults
Extendthelist
Extendthelistofedgedefinitionstoconstructagraph.
graph_module._extendEdgeDefinitions(edgeDefinitions,relation1,relation2,...,relationN)
Inordertoaddmoreedgedefinitionstothegraphbeforecreatingthisfunctioncanbeusedtoaddmoredefinitionstotheinitiallist.
Parameters
edgeDefinitions(required)Alistofrelationdefinitionobjects.
relationX(required)Anobjectrepresentingadefinitionofonerelationinthegraph
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>directed_relation=graph_module._relation("lives_in","user","city");
arangosh>undirected_relation=graph_module._relation("knows","user","user");
arangosh>edgedefinitions=graph_module._edgeDefinitions(directed_relation);
arangosh>edgedefinitions=graph_module._extendEdgeDefinitions(undirected_relation);
showexecutionresults
Relation
Defineadirectedrelation.
125
GraphM anagement
graph_module._relation(relationName,fromVertexCollections,toVertexCollections)
TherelationNamedefinesthenameofthisrelationandreferencestotheunderlyingedgecollection.ThefromVertexCollectionsisan
Arrayofdocumentcollectionsholdingthestartvertices.ThetoVertexCollectionsisanArrayofdocumentcollectionsholdingthetarget
vertices.RelationsareonlyallowedinthedirectionfromanycollectioninfromVertexCollectionstoanycollectionintoVertexCollections.
Parameters
relationName(required)Thenameoftheedgecollectionwheretheedgesshouldbestored.Willbecreatedifitdoesnotyetexist.
fromVertexCollections(required)Oneoralistofcollectionnames.Sourceverticesfortheedgeshavetobestoredinthese
collections.Collectionswillbecreatediftheydonotexist.
toVertexCollections(required)Oneoralistofcollectionnames.Targetverticesfortheedgeshavetobestoredinthesecollections.
Collectionswillbecreatediftheydonotexist.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._relation("has_bought",["Customer","Company"],["Groceries",
"Electronics"]);
showexecutionresults
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._relation("has_bought","Customer","Product");
showexecutionresults
Createagraph
Afterhavingintroducededgedefinitionsagraphcanbecreated.
Createagraph
graph_module._create(graphName,edgeDefinitions,orphanCollections)
Thecreationofagraphrequiresthenameofthegraphandadefinitionofitsedges.
Foreverytypeofedgedefinitionaconveniencemethodexiststhatcanbeusedtocreateagraph.Optionallyalistofvertexcollections
canbeadded,whicharenotusedinanyedgedefinition.Thesecollectionsarereferredtoasorphancollectionswithinthischapter.All
collectionsusedwithinthecreationprocessarecreatediftheydonotexist.
Parameters
graphName(required)Uniqueidentifierofthegraph
edgeDefinitions(optional)Listofrelationdefinitionobjects
orphanCollections(optional)Listofadditionalvertexcollectionnames
Examples
Createanemptygraph,edgedefinitionscanbeaddedatruntime:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph=graph_module._create("myGraph");
{[Graph]
}
Createagraphusinganedgecollection edgesandasinglevertexcollection vertices
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>varedgeDefinitions=[{collection:"edges","from":["vertices"],"to":[
"vertices"]}];
126
GraphM anagement
arangosh>graph=graph_module._create("myGraph",edgeDefinitions);
{[Graph]
"edges":[ArangoCollection17934,"edges"(typeedge,statusloaded)],
"vertices":[ArangoCollection17932,"vertices"(typedocument,statusloaded)]
}
Createagraphwithedgedefinitionsandorphancollections:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph=graph_module._create("myGraph",
........>[graph_module._relation("myRelation",["male","female"],["male","female"])],
["sessions"]);
showexecutionresults
CompleteExampletocreateagraph
ExampleCall:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>varedgeDefinitions=graph_module._edgeDefinitions();
arangosh>graph_module._extendEdgeDefinitions(edgeDefinitions,
graph_module._relation("friend_of","Customer","Customer"));
arangosh>graph_module._extendEdgeDefinitions(
........>edgeDefinitions,graph_module._relation(
........>"has_bought",["Customer","Company"],["Groceries","Electronics"]));
arangosh>graph_module._create("myStore",edgeDefinitions);
showexecutionresults
alternativecall:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>varedgeDefinitions=graph_module._edgeDefinitions(
........>graph_module._relation("friend_of",["Customer"],["Customer"]),
graph_module._relation(
........>"has_bought",["Customer","Company"],["Groceries","Electronics"]));
arangosh>graph_module._create("myStore",edgeDefinitions);
showexecutionresults
Listavailablegraphs
Listallgraphs.
graph_module._list()
Listsallgraphnamesstoredinthisdatabase.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._list();
[]
Loadagraph
127
GraphM anagement
Getagraph
graph_module._graph(graphName)
Agraphcanberetrievedbyitsname.
Parameters
graphName(required)Uniqueidentifierofthegraph
Examples
Getagraph:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph=graph_module._graph("social");
showexecutionresults
Removeagraph
Removeagraph
graph_module._drop(graphName,dropCollections)
Agraphcanbedroppedbyitsname.Thiscandropallcollectionscontainedinthegraphaslongastheyarenotusedwithinothergraphs.
Todropthecollectionsonlybelongingtothisgraph,theoptionalparameterdrop-collectionshastobesettotrue.
Parameters
graphName(required)Uniqueidentifierofthegraph
dropCollections(optional)Defineifcollectionsshouldbedropped(default:false)
Examples
Dropagraphandkeepcollections:
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._drop("social");
true
arangosh>db._collection("female");
[ArangoCollection18046,"female"(typedocument,statusloaded)]
arangosh>db._collection("male");
[ArangoCollection18048,"male"(typedocument,statusloaded)]
arangosh>db._collection("relation");
[ArangoCollection18050,"relation"(typeedge,statusloaded)]
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>graph_module._drop("social",true);
true
arangosh>db._collection("female");
null
arangosh>db._collection("male");
null
arangosh>db._collection("relation");
null
Modifyagraphdefinitionduringruntime
Afteryouhavecreatedangraphitsdefinitionisnotimmutable.Youcanstilladd,deleteormodifyedgedefinitionsandvertexcollections.
128
GraphM anagement
Extendtheedgedefinitions
Addanotheredgedefinitiontothegraph
graph._extendEdgeDefinitions(edgeDefinition)
Extendstheedgedefinitionsofagraph.Ifanorphancollectionisusedinthisedgedefinition,itwillberemovedfromtheorphanage.If
theedgecollectionoftheedgedefinitiontoaddisalreadyusedinthegraphorusedinadifferentgraphwithdifferentfromand/orto
collectionsanerroristhrown.
Parameters
edgeDefinition(required)Therelationdefinitiontoextendthegraph
Examples
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vared2=graph_module._relation("myEC2",["myVC1"],["myVC3"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1]);
arangosh>graph._extendEdgeDefinitions(ed2);
Modifyanedgedefinition
M odifyanrelationdefinition
graph_module._editEdgeDefinition(edgeDefinition)
Editsonerelationdefinitionofagraph.Theedgedefinitionusedasargumentwillreplacetheexistingedgedefinitionofthegraphwhich
hasthesamecollection.VertexCollectionsofthereplacededgedefinitionthatarenotusedinthenewdefinitionwilltransformtoan
orphan.Orphansthatareusedinthisnewedgedefinitionwillbedeletedfromthelistoforphans.Othergraphswiththesameedge
definitionwillbemodified,too.
Parameters
edgeDefinition(required)Theedgedefinitiontoreplacetheexistingedgedefinitionwiththesameattributecollection.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>varoriginal=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>varmodified=graph_module._relation("myEC1",["myVC2"],["myVC3"]);
arangosh>vargraph=graph_module._create("myGraph",[original]);
arangosh>graph._editEdgeDefinitions(modified);
Deleteanedgedefinition
Deleteonerelationdefinition
graph_module._deleteEdgeDefinition(edgeCollectionName,dropCollection)
Deletesarelationdefinitiondefinedbytheedgecollectionofagraph.Ifthecollectionsdefinedintheedgedefinition(collection,from,to)
arenotusedinanotheredgedefinitionofthegraph,theywillbemovedtotheorphanage.
Parameters
edgeCollectionName(required)Nameofedgecollectionintherelationdefinition.
dropCollection(optional)Defineiftheedgecollectionshouldbedropped.Defaultfalse.
Examples
Removeanedgedefinitionbutkeeptheedgecollection:
arangosh>vargraph_module=require("@arangodb/general-graph")
129
GraphM anagement
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vared2=graph_module._relation("myEC2",["myVC1"],["myVC3"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1,ed2]);
arangosh>graph._deleteEdgeDefinition("myEC1");
arangosh>db._collection("myEC1");
[ArangoCollection24624,"myEC1"(typeedge,statusloaded)]
Removeanedgedefinitionanddroptheedgecollection:
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vared2=graph_module._relation("myEC2",["myVC1"],["myVC3"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1,ed2]);
arangosh>graph._deleteEdgeDefinition("myEC1",true);
arangosh>db._collection("myEC1");
null
ExtendvertexCollections
Eachgraphcanhaveanarbitraryamountofvertexcollections,whicharenotpartofanyedgedefinitionofthegraph.Thesecollections
arecalledorphancollections.Ifthegraphisextendedwithanedgedefinitionusingoneoftheorphans,itwillberemovedfromthesetof
orphancollectionautomatically.
Addavertexcollection
Addavertexcollectiontothegraph
graph._addVertexCollection(vertexCollectionName,createCollection)
Addsavertexcollectiontothesetoforphancollectionsofthegraph.Ifthecollectiondoesnotexist,itwillbecreated.Ifitisalready
usedbyanyedgedefinitionofthegraph,anerrorwillbethrown.
Parameters
vertexCollectionName(required)Nameofvertexcollection.
createCollection(optional)Iftruethecollectionwillbecreatedifitdoesnotexist.Default:true.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph");
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1]);
arangosh>graph._addVertexCollection("myVC3",true);
Gettheorphanedcollections
Getallorphancollections
graph._orphanCollections()
Returnsallvertexcollectionsofthegraphthatarenotusedinanyedgedefinition.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1]);
arangosh>graph._addVertexCollection("myVC3",true);
arangosh>graph._orphanCollections();
130
GraphM anagement
[
"myVC3"
]
Removeavertexcollection
Removeavertexcollectionfromthegraph
graph._removeVertexCollection(vertexCollectionName,dropCollection)
Removesavertexcollectionfromthegraph.Onlycollectionsnotusedinanyrelationdefinitioncanberemoved.Optionallythecollection
canbedeleted,ifitisnotusedinanyothergraph.
Parameters
vertexCollectionName(required)Nameofvertexcollection.
dropCollection(optional)Iftruethecollectionwillbedroppedifitisnotusedinanyothergraph.Default:false.
Examples
arangosh>vargraph_module=require("@arangodb/general-graph")
arangosh>vared1=graph_module._relation("myEC1",["myVC1"],["myVC2"]);
arangosh>vargraph=graph_module._create("myGraph",[ed1]);
arangosh>graph._addVertexCollection("myVC3",true);
arangosh>graph._addVertexCollection("myVC4",true);
arangosh>graph._orphanCollections();
arangosh>graph._removeVertexCollection("myVC3");
arangosh>graph._orphanCollections();
showexecutionresults
ManiuplatingVertices
Saveavertex
CreateanewvertexinvertexCollectionName
graph.vertexCollectionName.save(data)
Parameters
data(required)JSONdataofvertex.
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.male.save({name:"Floyd",_key:"floyd"});
showexecutionresults
Replaceavertex
ReplacesthedataofavertexincollectionvertexCollectionName
graph.vertexCollectionName.replace(vertexId,data,options)
Parameters
vertexId(required)_idattributeofthevertex
data(required)JSONdataofvertex.
131
GraphM anagement
options(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.male.save({neym:"Jon",_key:"john"});
arangosh>graph.male.replace("male/john",{name:"John"});
showexecutionresults
Updateavertex
UpdatesthedataofavertexincollectionvertexCollectionName
graph.vertexCollectionName.update(vertexId,data,options)
Parameters
vertexId(required)_idattributeofthevertex
data(required)JSONdataofvertex.
options(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.female.save({name:"Lynda",_key:"linda"});
arangosh>graph.female.update("female/linda",{name:"Linda",_key:"linda"});
showexecutionresults
Removeavertex
RemovesavertexincollectionvertexCollectionName
graph.vertexCollectionName.remove(vertexId,options)
Additionallyremovesallingoingandoutgoingedgesofthevertexrecursively(seeedgeremove).
Parameters
vertexId(required)_idattributeofthevertex
options(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.male.save({name:"Kermit",_key:"kermit"});
arangosh>db._exists("male/kermit")
arangosh>graph.male.remove("male/kermit")
arangosh>db._exists("male/kermit")
showexecutionresults
ManipulatingEdges
Saveanewedge
132
GraphM anagement
CreatesanedgefromvertexfromtovertextoincollectionedgeCollectionName
graph.edgeCollectionName.save(from,to,data,options)
Parameters
from(required)_idattributeofthesourcevertex
to(required)_idattributeofthetargetvertex
data(required)JSONdataoftheedge
options(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save("male/bob","female/alice",{type:"married",_key:
"bobAndAlice"});
showexecutionresults
Ifthecollectionsoffromandtoarenotdefinedinanedgedefinitionofthegraph,theedgewillnotbestored.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save(
........>"relation/aliceAndBob",
........>"female/alice",
........>{type:"married",_key:"bobAndAlice"});
[ArangoError1906:invalidedgebetweenrelation/aliceAndBobandfemale/alice.Doesn't
conformtoanyedgedefinition]
Replaceanedge
ReplacesthedataofanedgeincollectionedgeCollectionName.Notethat _fromand _toaremandatory.
graph.edgeCollectionName.replace(edgeId,data,options)
Parameters
edgeId(required)_idattributeoftheedge
data(required)JSONdataoftheedge
options(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save("female/alice","female/diana",{typo:"nose",_key:
"aliceAndDiana"});
arangosh>graph.relation.replace("relation/aliceAndDiana",{type:"knows",_from:
"female/alice",_to:"female/diana"});
showexecutionresults
Updateanedge
UpdatesthedataofanedgeincollectionedgeCollectionName
graph.edgeCollectionName.update(edgeId,data,options)
Parameters
133
GraphM anagement
edgeId(required)_idattributeoftheedge
data(required)JSONdataoftheedge
options(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save("female/alice","female/diana",{type:"knows",_key:
"aliceAndDiana"});
arangosh>graph.relation.update("relation/aliceAndDiana",{type:"quarreled",_key:
"aliceAndDiana"});
showexecutionresults
Removeanedge
RemovesanedgeincollectionedgeCollectionName
graph.edgeCollectionName.remove(edgeId,options)
Ifthisedgeisusedasavertexbyanotheredge,theotheredgewillberemoved(recursively).
Parameters
edgeId(required)_idattributeoftheedge
options(optional)Seecollectiondocumentation
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>graph.relation.save("female/alice","female/diana",{_key:"aliceAndDiana"});
arangosh>db._exists("relation/aliceAndDiana")
arangosh>graph.relation.remove("relation/aliceAndDiana")
arangosh>db._exists("relation/aliceAndDiana")
showexecutionresults
Connectedges
Getallconnectingedgesbetween2groupsofverticesdefinedbytheexamples
graph._getConnectingEdges(vertexExample,vertexExample2,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.
Parameters
vertexExample1(optional)SeeDefinitionofexamples
vertexExample2(optional)SeeDefinitionofexamples
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
edgeExamples:Filtertheedges,seeDefinitionofexamples
edgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.
vertex1CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertex
steps.
vertex2CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertex
steps.
Examples
Arouteplannerexample,allconnectingedgesbetweencapitals.
134
GraphM anagement
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._getConnectingEdges({isCapital:true},{isCapital:true});
[]
135
GraphFunctions
GraphFunctions
Thischapterdescribesvariousfunctionsonagraph.Alotoftheseacceptavertex(oredge)exampleasparameterasdefinedinthenext
section.
ExampleswillexplaintheAPIonthethecitygraph:
Definitionofexamples
Formanyofthefollowingfunctionsexamplescanbepassedinasaparameter.Examplesareusedtofiltertheresultsetforobjectsthat
matchtheconditions.Theseexamplescanhavethefollowingvalues:
null,thereisnomatchingexecutedallfoundresultsarevalid.
Astring,onlyresultsarereturned,which_idequalthevalueofthestring
Anexampleobject,definingasetofattributes.Onlyresultshavingtheseattributesarematched.
Alistcontainingexampleobjectsand/orstrings.Allresultsmatchingatleastoneoftheelementsinthelistarereturned.Getvertices
fromedges.
Getvertexfromofanedge
Getthesourcevertexofanedge
graph._fromVertex(edgeId)
Returnsthevertexdefinedwiththeattribute_fromoftheedgewithedgeIdasits_id.
Parameters
edgeId(required)_idattributeoftheedge
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
136
GraphFunctions
arangosh>vargraph=examples.loadGraph("social");
arangosh>varany=require("@arangodb").db.relation.any();
arangosh>graph._fromVertex("relation/"+any._key);
showexecutionresults
Getvertextoofanedge
Getthetargetvertexofanedge
graph._toVertex(edgeId)
Returnsthevertexdefinedwiththeattribute_tooftheedgewithedgeIdasits_id.
Parameters
edgeId(required)_idattributeoftheedge
Examples
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("social");
arangosh>varany=require("@arangodb").db.relation.any();
arangosh>graph._toVertex("relation/"+any._key);
showexecutionresults
_neighbors
Getallneighborsoftheverticesdefinedbytheexample
graph._neighbors(vertexExample,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.Thecomplexityof
thismethodisO(n*m^x)withnbeingtheverticesdefinedbytheparametervertexExamplex,mtheaverageamountofneighborsandx
themaximaldepths.HencethedefaultcallwouldhaveacomplexityofO(n*m);
Parameters
vertexExample(optional)SeeDefinitionofexamples
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).
edgeExamples:Filtertheedges,seeDefinitionofexamples
neighborExamples:Filtertheneighborvertices,seeDefinitionofexamples
edgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.
vertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredontheintermediatevertexsteps.
minDepth:Definestheminimalnumberofintermediatestepstoneighbors(defaultis1).
maxDepth:Definesthemaximalnumberofintermediatestepstoneighbors(defaultis1).
Examples
Arouteplannerexample,allneighborsofcapitals.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._neighbors({isCapital:true});
showexecutionresults
Arouteplannerexample,alloutboundneighborsofHamburg.
137
GraphFunctions
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._neighbors('germanCity/Hamburg',{direction:'outbound',maxDepth:2});
showexecutionresults
_commonNeighbors
Getallcommonneighborsoftheverticesdefinedbytheexamples.
graph._commonNeighbors(vertex1Example,vertex2Examples,optionsVertex1,optionsVertex2)
Thisfunctionreturnstheintersectionofgraph_module._neighbors(vertex1Example,optionsVertex1)and
graph_module._neighbors(vertex2Example,optionsVertex2).Forparameterdocumentationsee_neighbors.
ThecomplexityofthismethodisO(n*m^x)withnbeingthemaximalamountofverticesdefinedbytheparametersvertexExamples,m
theaverageamountofneighborsandxthemaximaldepths.HencethedefaultcallwouldhaveacomplexityofO(n*m);
Examples
Arouteplannerexample,allcommonneighborsofcapitals.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._commonNeighbors({isCapital:true},{isCapital:true});
showexecutionresults
Arouteplannerexample,allcommonoutboundneighborsofHamburgwithanyotherlocationwhichhaveamaximaldepthof2:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._commonNeighbors(
........>'germanCity/Hamburg',
........>{},
........>{direction:'outbound',maxDepth:2},
........>{direction:'outbound',maxDepth:2});
showexecutionresults
_countCommonNeighbors
Gettheamountofcommonneighborsoftheverticesdefinedbytheexamples.
graph._countCommonNeighbors(vertex1Example,vertex2Examples,optionsVertex1,optionsVertex2)
Similarto_commonNeighborsbutreturnscountinsteadoftheelements.
Examples
Arouteplannerexample,allcommonneighborsofcapitals.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>varexample={isCapital:true};
arangosh>varoptions={includeData:true};
arangosh>graph._countCommonNeighbors(example,example,options,options);
showexecutionresults
138
GraphFunctions
Arouteplannerexample,allcommonoutboundneighborsofHamburgwithanyotherlocationwhichhaveamaximaldepthof2:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>varoptions={direction:'outbound',maxDepth:2,includeData:true};
arangosh>graph._countCommonNeighbors('germanCity/Hamburg',{},options,options);
showexecutionresults
_commonProperties
Gettheverticesofthegraphthatsharecommonproperties.
graph._commonProperties(vertex1Example,vertex2Examples,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertex1Exampleand
vertex2Example.
ThecomplexityofthismethodisO(n)withnbeingthemaximalamountofverticesdefinedbytheparametersvertexExamples.
Parameters
vertex1Examples(optional)Filterthesetofsourcevertices,seeDefinitionofexamples
vertex2Examples(optional)Filterthesetofverticescomparedto,seeDefinitionofexamples
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
vertex1CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbesearchedforsourcevertices.
vertex2CollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbesearchedforcomparevertices.
ignoreProperties:Oneoralistofattributenamesofadocumentthatshouldbeignored.
Examples
Arouteplannerexample,alllocationswiththesameproperties:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._commonProperties({},{});
showexecutionresults
Arouteplannerexample,allcitieswhichsharesamepropertiesexceptforpopulation.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._commonProperties({},{},{ignoreProperties:'population'});
showexecutionresults
_countCommonProperties
Gettheamountofverticesofthegraphthatsharecommonproperties.
graph._countCommonProperties(vertex1Example,vertex2Examples,options)
Similarto_commonPropertiesbutreturnscountinsteadoftheobjects.
Examples
Arouteplannerexample,alllocationswiththesameproperties:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
139
GraphFunctions
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._countCommonProperties({},{});
showexecutionresults
Arouteplannerexample,allGermancitieswhichsharesamepropertiesexceptforpopulation.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._countCommonProperties({},{},{vertex1CollectionRestriction:
'germanCity',
........>vertex2CollectionRestriction:'germanCity',ignoreProperties:'population'});
showexecutionresults
_paths
The_pathsfunctionreturnsallpathsofagraph.
graph._paths(options)
Thisfunctiondeterminesallavailablepathsinagraph.
ThecomplexityofthismethodisO(n*n*m)withnbeingtheamountofverticesinthegraphandmtheaverageamountofconnected
edges;
Parameters
options(optional)Anobjectcontainingoptions,seebelow:
direction:Thedirectionoftheedges.Possiblevaluesareany,inboundandoutbound(default).
followCycles(optional):Ifsettotruethequeryfollowscyclesinthegraph,defaultisfalse.
minLength(optional):Definestheminimallengthapathmusthavetobereturned(defaultis0).
maxLength(optional):Definesthemaximallengthapathmusthavetobereturned(defaultis10).
Examples
Returnallpathsofthegraph"social":
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("social");
arangosh>g._paths();
showexecutionresults
Returnallinboundpathsofthegraph"social"withamaximallengthof1andaminimallengthof2:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("social");
arangosh>g._paths({direction:'inbound',minLength:1,maxLength:2});
showexecutionresults
_shortestPath
The_shortestPathfunctionreturnsallshortestpathsofagraph.
graph._shortestPath(startVertexExample,endVertexExample,options)
140
GraphFunctions
Thisfunctiondeterminesallshortestpathsinagraph.Thefunctionacceptsanid,anexample,alistofexamplesorevenanempty
exampleasparameterforstartandendvertex.Thelengthofapathisbydefaulttheamountofedgesfromonestartvertextoanend
vertex.Theoptionweightallowstheusertodefineanedgeattributerepresentingthelength.
Parameters
startVertexExample(optional)AnexampleforthedesiredstartVertices(seeDefinitionofexamples).
endVertexExample(optional)AnexampleforthedesiredendVertices(seeDefinitionofexamples).
options(optional)Anobjectcontainingoptions,seebelow:
direction:Thedirectionoftheedgesasastring.Possiblevaluesareoutbound,inboundandany(default).
edgeCollectionRestriction:Oneormultipleedgecollectionnames.Onlyedgesfromthesecollectionswillbeconsideredforthe
path.
startVertexCollectionRestriction:Oneormultiplevertexcollectionnames.Onlyverticesfromthesecollectionswillbe
consideredasstartvertexofapath.
endVertexCollectionRestriction:Oneormultiplevertexcollectionnames.Onlyverticesfromthesecollectionswillbeconsidered
asendvertexofapath.
weight:Thenameoftheattributeoftheedgescontainingthelengthasastring.
defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthis
defaultisusedaslength.IfnodefaultissuppliedthedefaultwouldbepositiveInfinitysothepathcouldnotbecalculated.
Examples
Arouteplannerexample,shortestpathfromallgermantoallfrenchcities:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>g._shortestPath({},{},{weight:'distance',endVertexCollectionRestriction:
'frenchCity',
........>startVertexCollectionRestriction:'germanCity'});
showexecutionresults
Arouteplannerexample,shortestpathfromHamburgandColognetoLyon:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>g._shortestPath([{_id:'germanCity/Cologne'},{_id:'germanCity/Munich'}],
'frenchCity/Lyon',
........>{weight:'distance'});
showexecutionresults
_distanceTo
The_distanceTofunctionreturnsallpathsandtheredistancewithinagraph.
graph._distanceTo(startVertexExample,endVertexExample,options)
Thisfunctionisawrapperofgraph._shortestPath.Itdoesnotreturntheactualpathbutonlythedistancebetweentwovertices.
Examples
Arouteplannerexample,shortestdistancefromallgermantoallfrenchcities:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>g._distanceTo({},{},{weight:'distance',endVertexCollectionRestriction:
'frenchCity',
........>startVertexCollectionRestriction:'germanCity'});
141
GraphFunctions
showexecutionresults
Arouteplannerexample,shortestdistancefromHamburgandColognetoLyon:
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>varg=examples.loadGraph("routeplanner");
arangosh>g._distanceTo([{_id:'germanCity/Cologne'},{_id:'germanCity/Munich'}],
'frenchCity/Lyon',
........>{weight:'distance'});
showexecutionresults
_absoluteEccentricity
Gettheeccentricityoftheverticesdefinedbytheexamples.
graph._absoluteEccentricity(vertexExample,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.
Parameters
vertexExample(optional)Filterthevertices,seeDefinitionofexamples
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).
edgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.
startVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredforsourcevertices.
endVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredfortargetvertices.
weight:Thenameoftheattributeoftheedgescontainingtheweight.
defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthis
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheeccentricity
cannotbecalculated.
Examples
Arouteplannerexample,theabsoluteeccentricityofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteEccentricity({});
showexecutionresults
Arouteplannerexample,theabsoluteeccentricityofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteEccentricity({},{weight:'distance'});
showexecutionresults
Arouteplannerexample,theabsoluteeccentricityofallcitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteEccentricity({},{startVertexCollectionRestriction:
'germanCity',
........>direction:'outbound',weight:'distance'});
showexecutionresults
142
GraphFunctions
_eccentricity
Getthenormalizedeccentricityoftheverticesdefinedbytheexamples.
graph._eccentricity(vertexExample,options)
Similarto_absoluteEccentricitybutreturnsanormalizedresult.
Examples
Arouteplannerexample,theeccentricityofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._eccentricity();
showexecutionresults
Arouteplannerexample,theweightedeccentricity.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._eccentricity({weight:'distance'});
showexecutionresults
_absoluteCloseness
Gettheclosenessoftheverticesdefinedbytheexamples.
graph._absoluteCloseness(vertexExample,options)
Thefunctionacceptsanid,anexample,alistofexamplesorevenanemptyexampleasparameterforvertexExample.
Parameters
vertexExample(optional)Filterthevertices,seeDefinitionofexamples
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).
edgeCollectionRestriction:Oneoralistofedge-collectionnamesthatshouldbeconsideredtobeonthepath.
startVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredforsourcevertices.
endVertexCollectionRestriction:Oneoralistofvertex-collectionnamesthatshouldbeconsideredfortargetvertices.
weight:Thenameoftheattributeoftheedgescontainingtheweight.
defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthis
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheclosenesscan
notbecalculated.
Examples
Arouteplannerexample,theabsoluteclosenessofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteCloseness({});
showexecutionresults
Arouteplannerexample,theabsoluteclosenessofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
143
GraphFunctions
arangosh>graph._absoluteCloseness({},{weight:'distance'});
showexecutionresults
Arouteplannerexample,theabsoluteclosenessofallGermanCitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteCloseness({},{startVertexCollectionRestriction:'germanCity',
........>direction:'outbound',weight:'distance'});
showexecutionresults
_closeness
Getthenormalizedclosenessofgraphsvertices.
graph._closeness(options)
Similarto_absoluteClosenessbutreturnsanormalizedvalue.
Examples
Arouteplannerexample,thenormalizedclosenessofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._closeness();
showexecutionresults
Arouteplannerexample,theclosenessofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._closeness({weight:'distance'});
showexecutionresults
Arouteplannerexample,theclosenessofallcitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._closeness({direction:'outbound',weight:'distance'});
showexecutionresults
_absoluteBetweenness
Getthebetweennessofallverticesinthegraph.
graph._absoluteBetweenness(vertexExample,options)
Parameters
vertexExample(optional)Filterthevertices,seeDefinitionofexamples
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).
weight:Thenameoftheattributeoftheedgescontainingtheweight.
defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthis
144
GraphFunctions
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencethebetweeness
cannotbecalculated.
Examples
Arouteplannerexample,theabsolutebetweennessofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteBetweenness({});
showexecutionresults
Arouteplannerexample,theabsolutebetweennessofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteBetweenness({weight:'distance'});
{
}
Arouteplannerexample,theabsolutebetweennessofallcitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._absoluteBetweenness({direction:'outbound',weight:'distance'});
{
}
_betweenness
Getthenormalizedbetweennessofgraphsvertices.
graph_module._betweenness(options)
Similarto_absoluteBetweenessbutreturnsnormalizedvalues.
Examples
Arouteplannerexample,thebetweennessofalllocations.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._betweenness();
showexecutionresults
Arouteplannerexample,thebetweennessofalllocations.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._betweenness({weight:'distance'});
showexecutionresults
Arouteplannerexample,thebetweennessofallcitiesregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._betweenness({direction:'outbound',weight:'distance'});
145
GraphFunctions
showexecutionresults
_radius
Gettheradiusofagraph.
`
Parameters
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).
weight:Thenameoftheattributeoftheedgescontainingtheweight.
defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthis
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheradiuscan
notbecalculated.
Examples
Arouteplannerexample,theradiusofthegraph.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._radius();
1
Arouteplannerexample,theradiusofthegraph.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._radius({weight:'distance'});
1
Arouteplannerexample,theradiusofthegraphregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._radius({direction:'outbound',weight:'distance'});
1
_diameter
Getthediameterofagraph.
graph._diameter(graphName,options)
Parameters
options(optional)Anobjectdefiningfurtheroptions.Canhavethefollowingvalues:
direction:Thedirectionoftheedges.Possiblevaluesareoutbound,inboundandany(default).
weight:Thenameoftheattributeoftheedgescontainingtheweight.
defaultWeight:Onlyusedwiththeoptionweight.Ifanedgedoesnothavetheattributenamedasdefinedinoptionweightthis
defaultisusedasweight.Ifnodefaultissuppliedthedefaultwouldbepositiveinfinitysothepathandhencetheradiuscan
notbecalculated.
Examples
146
GraphFunctions
Arouteplannerexample,thediameterofthegraph.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._diameter();
1
Arouteplannerexample,thediameterofthegraph.Thisconsiderstheactualdistances.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._diameter({weight:'distance'});
1
Arouteplannerexample,thediameterofthegraphregardingonlyoutboundpaths.
arangosh>varexamples=require("@arangodb/graph-examples/example-graph.js");
arangosh>vargraph=examples.loadGraph("routeplanner");
arangosh>graph._diameter({direction:'outbound',weight:'distance'});
1
147
SmartGraphs
SmartGraphs
ThisfeatureisonlyavailableintheEnterpriseEdition.
Thischapterdescribesthesmart-graphmodule.Itenablesyoutomanagegraphsatscale,itwillgiveavastperformancebenefitforall
graphsshardedinanArangoDBCluster.Onasingleserverthisfeatureispointless,henceitisonlyavailableinaclustermode.Interms
ofqueryingthereisnodifferencebetweensmartandGeneralGraphs.Theformerareatransparentreplacementforthelatter.Sofor
queryingthegraphpleaserefertoAQLGraphOperationsandGraphFunctionssections.Theoptimizeriscleverenoughtoidentifyif
weareonaSmartGraphornot.
Thedifferenceisonlyinthemanagementsection:creatingandmodifyingtheunderlyingcollectionsofthegraph.ForadetailedAPI
referencepleaserefertoSmartGraphM anagement.
Whatmakesagraphsmart?
M ostgraphshaveonefeaturethatdividestheentiregraphintoseveralsmallersubgraphs.Thesesubgraphshavealargeamountofedges
thatonlyconnectverticesinthesamesubgraphandonlyhavefewedgesconnectingverticesfromothersubgraphs.Examplesforthese
graphsare:
SocialNetworks
Typicallythefeaturehereistheregion/countryuserslivein.Everyusertypicalliyhasmorecontactsinthesameregion/country
thenshehasinotherregions/countries
TransportSystems
Forthosealsothefeatureistheregion/country.Youhavemanylocaltransportionbutonlyfewaccrosscountries.
E-Commerce
Inthiscaseprobablythecategoryofproductsisagoodfeature.Oftenproductsofthesamecategoryareboughttogether.
Ifthisfeatureisknown,SmartGraphscanmakeuseifit.WhencreatingaSmartGraphyouhavetodefineasmartAttribute,whichisthe
nameofanattributestoredineveryvertex.Thegraphwillthanbeautomaticallyshardedinsuchawaythatallverticeswiththesame
valuearestoredonthesamephysicalmachine,alledgesconnectingverticeswithidenticalsmartAttributevaluesarestoredonthis
machineaswell.Duringquerytimethequeryoptimizerandthequeryexecutorbothknowforeverydocumentexactlywhereitisstored
andcantherebyminimizenetworkoverhead.Everythingthatcanbecomputedlocallywillbecomputedlocally.
BenefitsofSmartGraphs
Becauseoftheabovedescribedguaranteedsharding,theperformanceofqueriesthatonlycoveronesubgraphhaveaperformancealmost
equaltoanonlylocalcomputation.Queriesthatcovermorethanonesubgraphrequiresomenetworkoverhead.Themoresubgraphsare
touchedthemorenetworkcostwillapply.HowevertheoverallperformanceisneverworsethanthesamequeryonaGeneralGraph.
Gettingstarted
FirstofallSmartGraphscannotuseexistingcollections,whenswitchingtoSmartGraphfromanexistingdatasetyouhavetoimportthe
dataintoafreshSmartGraph.Thisswitchcanbeeasilyachievedwitharangodumpandarangorestore.Theonlythingyouhavetochange
inthispipelineisthatyoucreatethenewcollectionswiththeSmartGraphbeforestarting arangorestore.
Createagraph
IncomparisontoGeneralGraphwehavetoaddmoreoptionswhencreatingthegraph.Thetwooptions smartGraphAttributeand
numberOfShardsarerequiredandcannotbemodifedlater.
arangosh>vargraph_module=require("@arangodb/smart-graph");
arangosh>vargraph=graph_module._create("myGraph",[],[],{smartGraphAttribute:"region",numberOfShards:9});
arangosh>graph;
[SmartGraphmyGraphEdgeDefinitions:[]VertexCollections:[]]
Addsomevertexcollections
148
SmartGraphs
ThisisagainidenticaltoGeneralGraph.Themodulewillsetupcorrectshardingforallthesecollections.Note:Thecollectionshave
tobenew.
arangosh>graph._addVertexCollection("shop");
arangosh>graph._addVertexCollection("customer");
arangosh>graph._addVertexCollection("pet");
arangosh>graph;
[SmartGraphmyGraphEdgeDefinitions:[]VertexCollections:["shop","customer","pet"]]
DefinerelationsontheGraph
arangosh>varrel=graph_module._relation("isCustomer",["shop"],["customer"]);
arangosh>graph._extendEdgeDefinitions(rel);
arangosh>graph;
[SmartGraphmyGraphEdgeDefinitions:["isCustomer:[shop]->[customer]"]VertexCollections:["pet"]]
149
SmartGraphM anagement
SmartGraphManagement
ThischapterdescribestheJavaScriptinterfaceforcreatingandmodifyingSmartGraphs.AtfirstyouhavetonotethateverySmartGraph
isaspecializedversionofaGeneralGraph,whichmeansalloftheGeneralGraphfunctionalityisavailableonaSmartGraphaswell.The
majordifferenceofbothmodulesishandlingoftheunderlyingcollections,theGeneralGraphdoesnotenforceormaintainanysharding
ofthecollectionsandcanthereforcombinearbitrarysetsofexistingcollections.SmartGraphsenforceandrelyonaspecialshardingof
theunderlyingcollectionsandhencecanonlyworkwithcollectionsthatarecreatedthroughtheSmartGraphitself.Thisalsomeansthat
SmartGraphscannotbeoverlapping,acollectioncaneitherbeshardedforoneSmartGraphorfortheother.Ifyouneedtomakesurethat
allqueriescanbeexecutedwithSmartGraphperformance,justcreateonelargeSmartGraphcoveringeverythingandqueryitstatingthe
subsetofedgecollectionsexplicitly.TogenerallyunderstandtheconceptofthismodulepleasereadthechapteraboutGeneralGraph
M anagementfirst.Inthefollowingwewillonlydescribetheoverloadedfunctionality.Everythingelseworksidenticalinbothmodules.
Createagraph
AlsoSmartGraphsrequireedgerelationstobecreated,theformatoftherelationsisidentical.Theonlydifferenceisthatallcollections
usedwithintherelationstocreateanewSmartGraphcannotexistyet.TheyhavetobecreatedbytheGraphinordertoenforcethe
correctsharding.
Createagraph
graph_module._create(graphName,edgeDefinitions,orphanCollections,smartOptions)
ThecreationofagraphrequiresthenameandsomeSmartGraphoptions.DuetotheAPI edgeDefinitionsand orphanCollections
havetobegiven,butbothcanbeemptyarraysandcanbecreatedlater.The edgeDefinitionscanbecreatedusingtheconvenience
method _relationknownfromthe general-graphmodule,whichisalsoavailablehere. orphanCollectionsagainisjustalistof
additionalvertexcollectionswhicharenotyetconnectedviaedgesbutshouldfollowthesameshardingtobeconnectedlateron.All
collectionsusedwithinthecreationprocessarenewlycreated.Theprocesswillfailifoneofthemalreadyexists.Allnewlycreated
collectionswillimmediatelybedroppedagaininthefailedcase.
Parameters
graphName(required)Uniqueidentifierofthegraph
edgeDefinitions(required)Listofrelationdefinitionobjects,maybeempty
orphanCollections(required)Listofadditionalvertexcollectionnames,maybeempty
smartOptions(required)AJSONobjecthavingthefollowingkeys:
numberOfShards(required)Thenumberofshardsthatwillbecreatedforeachcollection.Tomaintainthecorrectshardingall
collectionsneedanidenticalnumberofshards.Thiscannotbemodifiedaftercreationofthegraph.
smartGraphAttribute(required)Theattributethatwillbeusedforsharding.Allverticesarerequiredtohavethisattributeset
andithastobeastring.Edgesderivetheattributefromtheirconnectedvertices.
Examples
Createanemptygraph,edgedefinitionscanbeaddedatruntime:
arangosh>vargraph_module=require("@arangodb/smart-graph");
arangosh>vargraph=graph_module._create("myGraph",[],[],{smartGraphAttribute:"region",numberOfShards:9});
[SmartGraphmyGraphEdgeDefinitions:[]VertexCollections:[]]
Createagraphusinganedgecollection edgesandasinglevertexcollection vertices
arangosh>vargraph_module=require("@arangodb/smart-graph");
arangosh>varedgeDefinitions=[graph_module._relation("edges","vertices","vertices")];
arangosh>vargraph=graph_module._create("myGraph",edgeDefinitions,[],{smartGraphAttribute:"region",numberOfShards:9});
[SmartGraphmyGraphEdgeDefinitions:["edges:[vertices]->[vertices]"]VertexCollections:[]]
Createagraphwithedgedefinitionsandorphancollections:
arangosh>vargraph_module=require("@arangodb/smart-graph");
arangosh>varedgeDefinitions=[graph_module._relation("myRelation",["male","female"],["male","female"])];
150
SmartGraphM anagement
arangosh>vargraph=graph_module._create("myGraph",edgeDefinitions,["sessions"],{smartGraphAttribute:"region",numberOfSh
ards:9});
[GraphmyGraphEdgeDefinitions:[
"myRelation:[female,male]->[female,male]"
]VertexCollections:[
"sessions"
]]
Modifyagraphdefinitionduringruntime
AfteryouhavecreatedaSmartGraphitsdefinitionisalsonotimmutable.Youcanstilladdorremoverelations.Thisisagainidenticalto
GeneralGraphs.Howeverthereisoneimportantdifference:Youcanonlyaddcollectionsthateitherdonotexist,orthathavebeen
createdbythisgraphearlier.Thelatercanbeachievedifyouforexampleremoveanorphancollectionfromthisgraph,withoutdropping
thecollectionitself.Thanaftersometimeyoudecidetoadditagain,itcanbeused.Thisisbecausetheenforcedshardingisstillapplied
tothisvertexcollection,henceitissuitabletobeaddedagain.
Removeavertexcollection
Removeavertexcollectionfromthegraph
graph._removeVertexCollection(vertexCollectionName,dropCollection)
InmostcasesthisfunctionworksidenticallytotheGeneralGraphone.Butthereisonespecialcase:Thefirstvertexcollectionaddedto
thegraph(eitherorphanorwithinarelation)definestheshardingforallcollectionswithinthegraph.Thiscollectioncanneverbe
removedfromthegraph.
Parameters
vertexCollectionName(required)Nameofvertexcollection.
dropCollection(optional)Iftruethecollectionwillbedroppedifitisnotusedinanyothergraph.Default:false.
Examples
Thefollowingexampleshowsthatyoucannotdroptheinitialcollection.Youhavetodropthecompletegraph.Ifyoujustwanttogetrid
ofthedata truncateit.
arangosh>vargraph_module=require("@arangodb/smart-graph")
arangosh>varrelation=graph_module._relation("edges","vertices","vertices");
arangosh>vargraph=graph_module._create("myGraph",[relation],["other"],{smartGraphAttribute:"region",numberOfShards:
9});
arangosh>graph._orphanCollections();
[
"other"
]
arangosh>graph._deleteEdgeDefinition("edges");
arangosh>graph._orphanCollections();
[
"vertices",
"other"
]
arangosh>graph._removeVertexCollection("other");
arangosh>graph._orphanCollections();
[
"vertices"
]
arangosh>graph._removeVertexCollection("vertices");
ArangoError4002:cannotdropthissmartcollection
151
Traversals
Traversals
ArangoDBprovidesseveralwaystoquerygraphdata.Verysimpleoperationscanbecomposedwiththelow-leveledgemethodsedges,
inEdges,andoutEdgesforedgecollections.Theseworkonnamedandanonymousgraphs.Formorecomplexoperations,ArangoDB
providespredefinedtraversalobjects.
AlsoTraversalshavebeenaddedtoAQL.PleasereadthechapteraboutAQLtraversersalsbeforeyoucontinuereadinghere.M ostofthe
traversalcasesarecoveredbyAQLandwillbeexecutedinanoptimizedway.Onlyifthelogicforyouristoocomplextobedefined
usingAQLfiltersyoucanusethetraversalobjectdefinedherewhichgivesyoucompleteprogrammaticaccesstothedata.
Foranyofthefollowingexamples,we'llbeusingtheexamplecollectionsvande,populatedwithcontinents,countriesandcapitalsdata
listedbelow(seeExampleData).
StartingfromScratch
ArangoDBprovidestheedges,inEdges,andoutEdgesmethodsforedgecollections.Thesemethodscanbeusedtoquicklydetermineifa
vertexisconnectedtoothervertices,andwhich.ThisfunctionalitycanbeexploitedtowriteverysimplegraphqueriesinJavaScript.
Forexample,todeterminewhichedgesarelinkedtotheworldvertex,wecanuseinEdges:
db.e.inEdges('v/world').forEach(function(edge){
require("@arangodb").print(edge._from,"->",edge.type,"->",edge._to);
});
inEdgeswillgiveusallingoingedgesforthespecifiedvertexv/world.TheresultisaJavaScriptarraythatwecaniterateoverandprint
theresults:
v/continent-africa->is-in->v/world
v/continent-south-america->is-in->v/world
v/continent-asia->is-in->v/world
v/continent-australia->is-in->v/world
v/continent-europe->is-in->v/world
v/continent-north-america->is-in->v/world
Note:edges,inEdges,andoutEdgesreturnanarrayofedges.Ifwewanttoretrievethelinkedvertices,wecanuseeachedges'_fromand
_toattributesasfollows:
db.e.inEdges('v/world').forEach(function(edge){
require("@arangodb").print(db._document(edge._from).name,"->",edge.type,"->",db._document(edge._to).name);
});
Weareusingthedocumentmethodfromthedbobjecttoretrievetheconnectedverticesnow.
Whilethismaybesufficientforone-levelgraphoperations,writingatraversalbyhandmaybecometoocomplexformulti-level
traversals.
152
UsingTraversalObjects
Gettingstarted
Touseatraversalobject,wefirstneedtorequirethetraversalmodule:
vartraversal=require("@arangodb/graph/traversal");
varexamples=require("@arangodb/graph-examples/example-graph.js");
examples.loadGraph("worldCountry");
Wethenneedtosetupaconfigurationforthetraversalanddetermineatwhichvertextostartthetraversal:
varconfig={
datasource:traversal.generalGraphDatasourceFactory("worldCountry"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.inboundExpander,
maxDepth:1
};
varstartVertex=db._document("v/world");
Note:ThestartVertexneedstobeadocument,notonlyadocumentid.
Wecanthencreateatraverserandstartthetraversalbycallingitstraversemethod.Notethattraverseneedsaresultobject,whichitcan
modifyinplace:
varresult={
visited:{
vertices:[],
paths:[]
}
};
vartraverser=newtraversal.Traverser(config);
traverser.traverse(result,startVertex);
Finally,wecanprintthecontentsoftheresultsobject,limitedtothevisitedvertices.Wewillonlyprintthenameandtypeofeach
visitedvertexforbrevity:
require("@arangodb").print(result.visited.vertices.map(function(vertex){
returnvertex.name+"("+vertex.type+")";
}));
Thefullscript,whichincludesallstepscarriedoutsofaristhus:
vartraversal=require("@arangodb/graph/traversal");
varconfig={
datasource:traversal.generalGraphDatasourceFactory("worldCountry"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.inboundExpander,
maxDepth:1
};
varstartVertex=db._document("v/world");
varresult={
visited:{
vertices:[],
paths:[]
}
};
vartraverser=newtraversal.Traverser(config);
traverser.traverse(result,startVertex);
153
UsingTraversalObjects
require("@arangodb").print(result.visited.vertices.map(function(vertex){
returnvertex.name+"("+vertex.type+")";
}));
Theresultisanarrayofverticesthatwerevisitedduringthetraversal,startingatthestartvertex(i.e.v/worldinourexample):
[
"World(root)",
"Africa(continent)",
"Asia(continent)",
"Australia(continent)",
"Europe(continent)",
"NorthAmerica(continent)",
"SouthAmerica(continent)"
]
Note:Theresultislimitedtoverticesdirectlyconnectedtothestartvertex.WeachievedthisbysettingthemaxDepthattributeto1.Not
settingitwouldreturnthefullarrayofvertices.
TraversalDirection
Fortheexamplescontainedinthismanual,we'llbestartingthetraversalsatvertexv/world.Verticesinourgraphareconnectedlikethis:
v/world<-is-in<-continent(Africa)<-is-in<-country(Algeria)<-is-in<-capital(Algiers)
Togetanymeaningfulresults,wemusttraversethegraphininboundorder.Thismeans,we'llbefollowingallincomingedgesoftoa
vertex.Inthetraversalconfiguration,wehavespecifiedthisviatheexpanderattribute:
varconfig={
...
expander:traversal.inboundExpander
};
Forothergraphs,wemightwanttotraverseviatheoutgoingedges.Forthis,wecanusetheoutboundExpander.Thereisalsoan
anyExpander,whichwillfollowbothoutgoingandincomingedges.Thisshouldbeusedwithcareandthetraversalshouldalwaysbe
limitedtoamaximumnumberofiterations(e.g.usingthemaxIterationsattribute)inordertoterminateatsomepoint.
Toinvokethedefaultoutboundexpanderforagraph,simplyusethepredefinedfunction:
varconfig={
...
expander:traversal.outboundExpander
};
Pleasenotetheoutboundexpanderwillnotproduceanyoutputfortheexamplesifwestillstartthetraversalatthev/worldvertex.
Still,wecanusetheoutboundexpanderifwestartsomewhereelseinthegraph,e.g.
vartraversal=require("@arangodb/graph/traversal");
varconfig={
datasource:traversal.generalGraphDatasourceFactory("world_graph"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.outboundExpander
};
varstartVertex=db._document("v/capital-algiers");
varresult={
visited:{
vertices:[],
paths:[]
}
};
154
UsingTraversalObjects
vartraverser=newtraversal.Traverser(config);
traverser.traverse(result,startVertex);
require("@arangodb").print(result.visited.vertices.map(function(vertex){
returnvertex.name+"("+vertex.type+")";
}));
Theresultis:
[
"Algiers(capital)",
"Algeria(country)",
"Africa(continent)",
"World(root)"
]
whichconfirmsthatnowwe'regoingoutbound.
TraversalStrategy
Depth-firsttraversals
Thevisitationorderofverticesisdeterminedbythestrategyandorderattributessetintheconfiguration.Wechosedepthfirstand
preorder,meaningthetraverserwillvisiteachvertexbeforehandlingconnectededges(pre-order),anddescendintoanyconnectededges
beforeprocessingotherverticesonthesamelevel(depth-first).
Let'sremovethemaxDepthattributenow.We'llnowbegettingallvertices(directlyandindirectlyconnectedtothestartvertex):
varconfig={
datasource:traversal.generalGraphDatasourceFactory("world_graph"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.inboundExpander
};
varresult={
visited:{
vertices:[],
paths:[]
}
};
vartraverser=newtraversal.Traverser(config);
traverser.traverse(result,startVertex);
require("@arangodb").print(result.visited.vertices.map(function(vertex){
returnvertex.name+"("+vertex.type+")";
}));
Theresultwillbealongerarray,assembledindepth-first,pre-orderorder.Foreachcontinentfound,thetraverserwilldescendintolinked
countries,andthenintothelinkedcapital:
[
"World(root)",
"Africa(continent)",
"Algeria(country)",
"Algiers(capital)",
"Angola(country)",
"Luanda(capital)",
"Botswana(country)",
"Gaborone(capital)",
"BurkinaFaso(country)",
"Ouagadougou(capital)",
...
]
155
UsingTraversalObjects
Let'sswitchtheorderattributefrompreordertopostorder.Thiswillmakethetraverservisitverticesafterallconnectedverticeswere
visited(i.e.mostdistantverticeswillbeemittedfirst):
[
"Algiers(capital)",
"Algeria(country)",
"Luanda(capital)",
"Angola(country)",
"Gaborone(capital)",
"Botswana(country)",
"Ouagadougou(capital)",
"BurkinaFaso(country)",
"Bujumbura(capital)",
"Burundi(country)",
"Yaounde(capital)",
"Cameroon(country)",
"N'Djamena(capital)",
"Chad(country)",
"Yamoussoukro(capital)",
"Coted'Ivoire(country)",
"Cairo(capital)",
"Egypt(country)",
"Asmara(capital)",
"Eritrea(country)",
"Africa(continent)",
...
]
Breadth-firsttraversals
Ifwegobacktopreorder,butchangethestrategytobreadth-firstandre-runthetraversal,we'llseethatthereturnorderchanges,and
itemsonthesamelevelwillbereturnedadjacently:
[
"World(root)",
"Africa(continent)",
"Asia(continent)",
"Australia(continent)",
"Europe(continent)",
"NorthAmerica(continent)",
"SouthAmerica(continent)",
"BurkinaFaso(country)",
"Burundi(country)",
"Cameroon(country)",
"Chad(country)",
"Algeria(country)",
"Angola(country)",
...
]
Note:Theorderofitemsreturnedforthesamelevelisundefined.Thisisbecausethereisnonaturalorderofedgesforavertexwith
multipleconnectededges.Toexplicitlysettheorderforedgesonthesamelevel,youcanspecifyanedgecomparatorfunctionwiththe
sortattribute:
varconfig={
...
sort:function(l,r){returnl._key<r._key?1:-1;}
...
};
Theargumentslandrareedgedocuments.Thiswilltraverseedgesofthesamevertexinbackward_keyorder:
[
"World(root)",
"SouthAmerica(continent)",
"NorthAmerica(continent)",
"Europe(continent)",
"Australia(continent)",
"Asia(continent)",
156
UsingTraversalObjects
"Africa(continent)",
"Ecuador(country)",
"Colombia(country)",
"Chile(country)",
"Brazil(country)",
"Bolivia(country)",
"Argentina(country)",
...
]
Note:Thisattributeonlyworksfortheusualexpanderstraversal.inboundExpander,traversal.outboundExpander,
traversal.anyExpanderandtheircorresponding"WithLabels"variants.Ifyouareusingcustomexpandersyouhavetoorganizethe
sortingwithinthespecifiedexpander.
WritingCustomVisitors
Sofarwehaveusedmuchofthetraverser'sdefaultfunctions.Thetraverserisveryconfigurableandmanyofthedefaultfunctionscanbe
overriddenwithcustomfunctionality.
Forexample,wehavebeenusingthedefaultvisitorfunction(whichisalwaysusediftheconfigurationdoesnotcontainthevisitor
attribute).Thedefaultvisitorfunctioniscalledforeachvertexinatraversal,andwillpushitintotheresult.Thisisthereasonwhythe
resultvariablelookeddifferentafterthetraversal,andneededtobeinitializedbeforethetraversalwasstarted.
Notethatthedefaultvisitor(named trackingVisitor)willaddeveryvisitedvertexintotheresult,includingthefullpathsfromthestart
vertex.Thisisusefulforlearninganddebuggingpurposes,butshouldbeavoidedinproductionbecauseitmightproduce(andcopy)huge
amountsofdata.Instead,onlythosedatashouldbecopiedintotheresultthatareactuallynecessary.
Thetraversercomeswiththefollowingpredefinedvisitors:
trackingVisitor:thisisthedefaultvisitor.Itwillcopyalldataofeachvisitedvertexplusthefullpathinformationintotheresult.
Thiscanbeslowiftheresultsetishugeorverticescontainalotofdata.
countingVisitor:thisisaverylightweightvisitor:allitdoesisincreaseacounterintheresultforeachvertexvisited.Vertexdataand
pathswillnotbecopiedintotheresult.
doNothingVisitor:ifnoactionshallbecarriedoutwhenavertexisvisited,thisvisitorcanbeemployed.Itwillnotdoanythingand
willthusbefast.Itcanbeusedforperformancecomparisonswithothervisitors.
Wecanalsowriteourownvisitorfunctionifwewantto.Thegeneralfunctionsignatureforvisitorfunctionsisasfollows:
varconfig={
...
visitor:function(config,result,vertex,path,connected){...}
};
Note:theconnectedparametervaluewillonlybesetifthetraversalorderissettopreorder-expander.Otherwise,thisparameterwon't
besetbythetraverser.
Visitorfunctionsarenotexpectedtoreturnanyvalues.Instead,theycanmodifytheresultvariable(e.g.bypushingthecurrentvertex
intoit),ordoanythingelse.Forexample,wecancreateasimplevisitorfunctionthatonlyprintsinformationaboutthecurrentvertexas
wetraverse:
varconfig={
datasource:traversal.generalGraphDatasourceFactory("world_graph"),
strategy:"depthfirst",
order:"preorder",
filter:traversal.visitAllFilter,
expander:traversal.inboundExpander,
visitor:function(config,result,vertex,path){
require("@arangodb").print("visitingvertex",vertex.name);
}
};
vartraverser=newtraversal.Traverser(config);
traverser.traverse(undefined,startVertex);
Towriteavisitorthatincrementsacountereachtimeavertexisvisited,wecouldwritethefollowingcustomvisitor:
157
UsingTraversalObjects
config.visitor=function(config,result,vertex,path,connected){
if(!result){
result={};
}
if(!result.hasOwnProperty('count')){
result.count=0;
}
++result.count;
}
Notethatsuchvisitorisalreadypredefined(it'sthecountingVisitordescribedabove).Itcanbeusedasfollows:
config.visitor=traversal.countingVisitor;
Anotherexampleofavisitorisonethatcollectsthe _idvaluesofallverticesvisited:
config.visitor=function(config,result,vertex,path,connected){
if(!result){
result={};
}
if(!result.hasOwnProperty("visited")){
result.visited={vertices:[]};
}
result.visited.vertices.push(vertex._id);
}
Whenthetraversalorderissettopreorder-expander,thetraverserwillpassafifthparametervalueintothevisitorfunction.This
parametercontainstheconnectededgesofthevisitedvertexasanarray.Thiscanbehandybecauseinthiscasethevisitorwillgetall
informationaboutthevertexandtheconnectededgestogether.
Forexample,thefollowingvisitorcanbeusedtoprintonlyleafnodes(thatdonothaveanyfurtherconnectededges):
config.visitor=function(config,result,vertex,path,connected){
if(connected&&connected.length===0){
require("@arangodb").print("foundaleaf-node:",vertex);
}
}
Notethatforthisvisitortowork,thetraversalorderattributeneedstobesettothevaluepreorder-expander.
FilteringVerticesandEdges
FilteringVertices
Sofarwehaveprintedorreturnedallverticesthatwerevisitedduringthetraversal.Thisisnotalwaysrequired.Iftheresultshallbe
restricttojustspecificvertices,wecanuseafilterfunctionforvertices.Itcanbedefinedbysettingthefilterattributeofatraversal
configuration,e.g.:
varconfig={
filter:function(config,vertex,path){
if(vertex.type!=='capital'){
return'exclude';
}
}
}
Theabovefilterfunctionwillexcludeallverticesthatdonothaveatypevalueofcapital.Thefilterfunctionwillbecalledforeachvertex
foundduringthetraversal.Itwillreceivethetraversalconfiguration,thecurrentvertex,andthefullpathfromthetraversalstartvertexto
thecurrentvertex.Thepathconsistsofanarrayofedges,andanarrayofvertices.Wecouldalsofiltereverythingbutcapitalsby
158
UsingTraversalObjects
checkingthelengthofthepathfromthestartvertextothecurrentvertex.Capitalswillhaveadistanceof3fromthev/worldstartvertex
(capital→is-in→country→is-in→continent→is-in→world):
varconfig={
...
filter:function(config,vertex,path){
if(path.edges.length<3){
return'exclude';
}
}
}
Note:Ifafilterfunctionreturnsnothing(orundefined),thecurrentvertexwillbeincluded,andallconnectededgeswillbefollowed.Ifa
filterfunctionreturnsexcludethecurrentvertexwillbeexcludedfromtheresult,andallstillallconnectededgeswillbefollowed.Ifa
filterfunctionreturnsprune,thecurrentvertexwillbeincluded,butnoconnectededgeswillbefollowed.
Forexample,thefollowingfilterfunctionwillnotdescendintoconnectededgesofcontinents,limitingthedepthofthetraversal.Still,
continentverticeswillbeincludedintheresult:
varconfig={
...
filter:function(config,vertex,path){
if(vertex.type==='continent'){
return'prune';
}
}
}
Itisalsopossibletocombineexcludeandprunebyreturninganarraywithbothvalues:
return['exclude','prune'];
FilteringEdges
Itispossibletoexcludecertainedgesfromthetraversal.Tofilteronedges,afilterfunctioncanbedefinedviatheexpandFilterattribute.
TheexpandFilterisafunctionwhichiscalledforeachedgeduringatraversal.
Itwillreceivethecurrentedge(edgevariable)andthevertexwhichtheedgeconnectsto(inthedirectionofthetraversal).Italsoreceives
thecurrentpathfromthestartvertexuptothecurrentvertex(excludingthecurrentedgeandthevertextheedgepointsto).
Ifthefunctionreturnstrue,theedgewillbefollowed.Ifthefunctionreturnsfalse,theedgewillnotbefollowed.Hereisaverysimple
customedgefilterfunctionimplementation,whichsimplyincludesedgesifthe(edges)pathlengthislessthan1,andwillexcludeany
otheredges.Thiswilleffectivelyterminatethetraversalafterthefirstlevelofedges:
varconfig={
...
expandFilter:function(config,vertex,edge,path){
return(path.edges.length<1);
}
};
WritingCustomExpanders
Theedgesconnectedtoavertexaredeterminedbytheexpander.Sofarwehaveusedadefaultexpander(thedefaultinboundexpanderto
beprecise).Thedefaultinboundexpandersimplyenumeratesallconnectedingoingedgesforavertex,basedontheedgecollection
specifiedinthetraversalconfiguration.
Thereisalsoadefaultoutboundexpander,whichwillenumerateallconnectedoutgoingedges.Finally,thereisananyexpander,which
willfollowbothingoingandoutgoingedges.
Ifconnectededgesmustbedeterminedinsomedifferentfashionforwhateverreason,acustomexpandercanbewrittenandregisteredby
settingtheexpanderattributeoftheconfiguration.Theexpanderfunctionsignatureisasfollows:
159
UsingTraversalObjects
varconfig={
...
expander:function(config,vertex,path){...}
}
Itistheexpander'sresponsibilitytoreturnalledgesandverticesdirectlyconnectedtothecurrentvertex(whichispassedviathevertex
variable).Thefullpathfromthestartvertexuptothecurrentvertexisalsosuppliedviathepathvariable.Anexpanderisexpectedto
returnanarrayofobjects,whichneedtohaveanedgeandavertexattributeeach.
Note:Ifyouwanttorelyonaparticularorderinwhichtheedgesaretraversed,youhavetosorttheedgesreturnedbyyourexpander
withinthecodeoftheexpander.Thefunctionstogetoutbound,inboundoranyedgesfromavertexdonotguaranteeanyparticularorder!
Acustomimplementationofaninboundexpandercouldlooklikethis(thisisanon-deterministicexpander,whichrandomlydecides
whetherornottoincludeconnectededges):
varconfig={
...
expander:function(config,vertex,path){
varconnected=[];
vardatasource=config.datasource;
datasource.getInEdges(vertex._id).forEach(function(edge){
if(Math.random()>=0.5){
connected.push({edge:edge,vertex:(edge._from)});
}
});
returnconnected;
}
};
Acustomexpandercanalsobeusedasanedgefilterbecauseithasfullcontroloverwhichedgeswillbereturned.
Followingaretwoexamplesofcustomexpandersthatpickedgesbasedonattributesoftheedgesandtheconnectedvertices.
Findingtheconnectededges/verticesbasedonanattributewhenintheconnectedvertices.Thegoalistofollowtheedgethatleadstothe
vertexwiththehighestvalueinthewhenattribute:
varconfig={
...
expander:function(config,vertex,path){
vardatasource=config.datasource;
//determinealloutgoingedges
varoutEdges=datasource.getOutEdges(vertex);
if(outEdges.length===0){
return[];
}
vardata=[];
outEdges.forEach(function(edge){
data.push({edge:edge,vertex:datasource.getInVertex(edge)});
});
//sortoutgoingverticesaccordingto"when"attributevalue
data.sort(function(l,r){
if(l.vertex.when===r.vertex.when){
return0;
}
return(l.vertex.when<r.vertex.when?1:-1);
});
//pickfirstvertexfound(withhighest"when"attributevalue)
return[data[0]];
}
...
};
Findingtheconnectededges/verticesbasedonanattributewhenintheedgeitself.Thegoalistopicktheoneedge(outofpotentially
many)thathasthehighestwhenattributevalue:
160
UsingTraversalObjects
varconfig={
...
expander:function(config,vertex,path){
vardatasource=config.datasource;
//determinealloutgoingedges
varoutEdges=datasource.getOutEdges(vertex);
if(outEdges.length===0){
return[];//returnanemptyarray
}
//sortalloutgoingedgesaccordingto"when"attribute
outEdges.sort(function(l,r){
if(l.when===r.when){
return0;
}
return(l.when<r.when?-1:1);
});
//returnfirstedge(theonewithhighest"when"value)
varedge=outEdges[0];
try{
varv=datasource.getInVertex(edge);
return[{edge:edge,vertex:v}];
}
catch(e){}
return[];
}
...
};
HandlingUniqueness
Graphsmaycontaincycles.Tobeontopofwhathappenswhenatraversalencountersavertexoranedgeithasalreadyvisited,thereare
configurationoptions.
Thedefaultconfigurationistovisiteveryvertex,regardlessofwhetheritwasalreadyvisitedinthesametraversal.However,edgeswill
bydefaultonlybefollowediftheyarenotalreadypresentinthecurrentpath.
Imaginethefollowinggraphwhichcontainsacycle:
A->B->C->A
WhenthetraversalfindstheedgefromCtoA,itwillbydefaultfollowit.Thisisbecausewehavenotseenthisedgeyet.Itwillalsovisit
vertexAagain.Thisisbecausebydefaultallverticeswillbevisited,regardlessofwhetheralreadyvisitedornot.
However,thetraversalwillnotagainfollowingtheoutgoingedgefromAtoB.ThisisbecausewealreadyhavetheedgefromAtoBinour
currentpath.
Thesedefaultsettingswillpreventinfinitetraversals.
Toadjusttheuniquenessforvisitingvertices,therearethefollowingoptionsforuniqueness.vertices:
"none":alwaysvisitavertices,regardlessofwhetheritwasalreadyvisitedornot
"global":visitavertexonlyifitwasnotvisitedinthetraversal
"path":visitavertexifitisnotincludedinthecurrentpath
Toadjusttheuniquenessforfollowingedges,therearethefollowingoptionsforuniqueness.edges:
"none":alwaysfollowanedge,regardlessofwhetheritwasfollowedbefore
"global":followanedgeonlyifitwasn'tfollowedinthetraversal
"path":followanedgeifitisnotincludedinthecurrentpath
Notethatuniquenesscheckingwillhavesomeeffectonbothruntimeandmemoryusage.Forexample,whenuniquenesschecksaresetto
"global",arraysofvisitedverticesandedgesmustbekeptinmemorywhilethetraversalisexecuted.Globaluniquenessshouldthusonly
beusedwhenatraversalisexpectedtovisitfewnodes.
161
UsingTraversalObjects
Intermsofruntime,turningoffuniquenesschecks(bysettingbothoptionsto"none")isthebestchoice,butitisonlysafeforgraphs
thatdonotcontaincycles.Whenuniquenesschecksaredeactivatedinagraphwithcycles,thetraversalmightnotabortinasensible
amountoftime.
Optimizations
Thereareafewoptionsformakingatraversalrunfaster.
Thebestoptionistomaketheamountofvisitedverticesandfollowededgesassmallaspossible.Thiscanbeachievedbywriting
customfilterandexpanderfunctions.Suchfunctionsshouldonlyincludeverticesofinterest,andonlyfollowedgesthatmightbe
interesting.
TraversaldepthcanalsobeboundedwiththeminDepthandmaxDepthoptions.
Anotherwaytospeeduptraversalsistowriteacustomvisitorfunction.Thedefaultvisitorfunction(trackingVisitor)willcopyevery
visitedvertexintotheresult.Ifverticescontainlotsofdata,thismightbeexpensive.Itisthereforerecommendedtoonlycopysuchdata
intotheresultthatisactuallyneeded.Thedefaultvisitorfunctionwillalsocopythefullpathtothevisiteddocumentintotheresult.
Thisisevenmoreexpensiveandshouldbeavoidedifpossible.
Ifthegoalofatraversalistoonlycountthenumberofvisitedvertices,theprefabcountingVisitor
willbemuchmoreefficientthanthedefaultvisitor.
Forgraphsthatareknowntonotcontainanycycles,uniquenesschecksshouldbeturnedoff.Thiscanachievedviatheuniqueness
configurationoptions.Notethatuniquenesschecksshouldnotbeturnedoffforgraphsthatareknowncontaincyclesorifthereisno
informationaboutthegraph'sstructure.
Bydefault,atraversalwillonlyprocessalimitednumberofvertices.Thisisprotecttheuserfromunintentionallyrunanever-ending
traversalonagraphwithcyclicdata.HowmanyverticeswillbeprocessedatmostisdeterminedbythemaxIterationsconfiguration
option.IfatraversalhitsthecapspecifiedbymaxIterations,itwillabortandthrowatoomanyiterationsexception.Ifthiserroris
encountered,themaxIterationsvalueshouldbeincreasedifitismadesurethattheothertraversalconfigurationparametersaresaneand
thetraversalwillabortnaturallyatsomepoint.
Finally,thebuildVerticesconfigurationoptioncanbesettofalsetoavoidlookingupandfullyconstructingvertexdata.Ifallthat'sneeded
fromverticesarethe_idor_keyattributes,thebuildverticesoptioncanbesettofalse.Ifvisitor,filterorexpandFilterfunctionsneedto
accessothervertexattributes,theoptionshouldnotbechanged.
ConfigurationOverview
Thissectionsummarizestheconfigurationattributesforthetraversalobject.Theconfigurationcanconsistofthefollowingattributes:
visitor:visitorfunctionforvertices.Itwillbecalledforallnon-excludedvertices.Thegeneralvisitorfunctionsignatureisfunction
(config,result,vertex,path).Ifthetraversalorderispreorder-expander,theconnectingedgesofthevisitedvertexwillbepassedas
thefifthparameter,extendingthefunctionsignatureto:function(config,result,vertex,path,edges).
Visitorfunctionsarenotexpectedtoreturnvalues,buttheymaymodifytheresultvariableasneeded(e.g.bypushingvertexdata
intotheresult).
expander:expanderfunctionthatisresponsibleforreturningedgesandverticesdirectlyconnectedtoavertex.Thefunction
signatureisfunction(config,vertex,path).Theexpanderfunctionisrequiredtoreturnanarrayofconnectionobjects,consistingof
anedgeandvertexattributeeach.Iftherearenoconnectingedges,theexpanderisexpectedtoreturnanemptyarray.
filter:vertexfilterfunction.Thefunctionsignatureisfunction(config,vertex,path).Itmayreturnoneofthefollowingvalues:
undefined:vertexwillbeincludedintheresultandconnectededgeswillbetraversed
"exclude":vertexwillnotbeincludedintheresultandconnectededgeswillbetraversed
"prune":vertexwillbeincludedintheresultbutconnectededgeswillnotbetraversed
["prune","exclude"]:vertexwillnotbeincludedintheresultandconnectededgeswillnotbereturned
expandFilter:filterfunctionappliedoneachedge/vertexcombinationdeterminedbytheexpander.Thefunctionsignatureisfunction
(config,vertex,edge,path).Thefunctionshouldreturntrueiftheedge/vertexcombinationshouldbeprocessed,andfalseifitshould
beignored.
sort:afilterfunctiontodeterminetheorderinwhichconnectededgesareprocessed.Thefunctionsignatureisfunction(l,r).The
functionisrequiredtoreturnoneofthefollowingvalues:
-1iflshouldhaveasortvaluelessthanr
162
UsingTraversalObjects
1iflshouldhaveahighersortvaluethanr
0iflandrhavethesamesortvalue
strategy:determinesthevisitationstrategy.Possiblevaluesaredepthfirstandbreadthfirst.
order:determinesthevisitationorder.Possiblevaluesarepreorder,postorder,andpreorder-expander.preorder-expanderisthe
sameaspreorder,exceptthatthesignatureofthevisitorfunctionwillchangeasdescribedabove.
itemOrder:determinestheorderinwhichconnectionsreturnedbytheexpanderwillbeprocessed.Possiblevaluesareforwardand
backward.
maxDepth:ifsettoavaluegreaterthan0,thiswilllimitthetraversaltothismaximumdepth.
minDepth:ifsettoavaluegreaterthan0,allverticesfoundonalevelbelowtheminDepthlevelwillnotbeincludedintheresult.
maxIterations:themaximumnumberofiterationsthatthetraversalisallowedtoperform.Itissensibletosetthisnumberso
unboundedtraversalswillterminateatsomepoint.
uniqueness:anobjectthatdefineshowrepeatedvisitationsofverticesshouldbehandled.Theuniquenessobjectcanhaveasubattributevertices,andasub-attributeedges.Eachsub-attributecanhaveoneofthefollowingvalues:
"none":nouniquenessconstraints
"path":elementisexcludedifitisalreadycontainedinthecurrentpath.Thissettingmaybesensibleforgraphsthatcontain
cycles(e.g.A→B→C→A).
"global":elementisexcludedifitwasalreadyfound/visitedatanypointduringthetraversal.
buildVertices:thisattributecontrolswhetherverticesencounteredduringthetraversalwillbelookedupinthedatabaseandwillbe
madeavailabletovisitor,filter,andexpandFilterfunctions.Bydefault,verticeswillbelookedupandmadeavailable.However,
therearesomespecialusecaseswhenfullyconstructingvertexobjectsisnotnecessaryandcanbeavoided.Forexample,ifa
traversalismeanttoonlycountthenumberofvisitedverticesbutdonotreadanydatafromvertices,thisoptionmightbesetto
true.
163
ExampleData
ExampleData
Thefollowingexamplesalluseavertexcollectionvandanedgecollectione.Thevertexcollectionvcontainscontinents,countries,and
capitals.Theedgecollectionecontainsconnectionsbetweencontinentsandcountries,andbetweencountriesandcapitals.
Tosetupthecollectionsandpopulatethemwithinitialdata,thefollowingscriptwasused:
db._create("v");
db._createEdgeCollection("e");
//vertices:rootnode
db.v.save({_key:"world",name:"World",type:"root"});
//vertices:continents
db.v.save({_key:"continent-africa",name:"Africa",type:"continent"});
db.v.save({_key:"continent-asia",name:"Asia",type:"continent"});
db.v.save({_key:"continent-australia",name:"Australia",type:"continent"});
db.v.save({_key:"continent-europe",name:"Europe",type:"continent"});
db.v.save({_key:"continent-north-america",name:"NorthAmerica",type:"continent"});
db.v.save({_key:"continent-south-america",name:"SouthAmerica",type:"continent"});
//vertices:countries
db.v.save({_key:"country-afghanistan",name:"Afghanistan",type:"country",code:"AFG"});
db.v.save({_key:"country-albania",name:"Albania",type:"country",code:"ALB"});
db.v.save({_key:"country-algeria",name:"Algeria",type:"country",code:"DZA"});
db.v.save({_key:"country-andorra",name:"Andorra",type:"country",code:"AND"});
db.v.save({_key:"country-angola",name:"Angola",type:"country",code:"AGO"});
db.v.save({_key:"country-antigua-and-barbuda",name:"AntiguaandBarbuda",type:"country",code:"ATG"});
db.v.save({_key:"country-argentina",name:"Argentina",type:"country",code:"ARG"});
db.v.save({_key:"country-australia",name:"Australia",type:"country",code:"AUS"});
db.v.save({_key:"country-austria",name:"Austria",type:"country",code:"AUT"});
db.v.save({_key:"country-bahamas",name:"Bahamas",type:"country",code:"BHS"});
db.v.save({_key:"country-bahrain",name:"Bahrain",type:"country",code:"BHR"});
db.v.save({_key:"country-bangladesh",name:"Bangladesh",type:"country",code:"BGD"});
db.v.save({_key:"country-barbados",name:"Barbados",type:"country",code:"BRB"});
db.v.save({_key:"country-belgium",name:"Belgium",type:"country",code:"BEL"});
db.v.save({_key:"country-bhutan",name:"Bhutan",type:"country",code:"BTN"});
db.v.save({_key:"country-bolivia",name:"Bolivia",type:"country",code:"BOL"});
db.v.save({_key:"country-bosnia-and-herzegovina",name:"BosniaandHerzegovina",type:"country",code:"BIH"});
db.v.save({_key:"country-botswana",name:"Botswana",type:"country",code:"BWA"});
db.v.save({_key:"country-brazil",name:"Brazil",type:"country",code:"BRA"});
db.v.save({_key:"country-brunei",name:"Brunei",type:"country",code:"BRN"});
db.v.save({_key:"country-bulgaria",name:"Bulgaria",type:"country",code:"BGR"});
db.v.save({_key:"country-burkina-faso",name:"BurkinaFaso",type:"country",code:"BFA"});
db.v.save({_key:"country-burundi",name:"Burundi",type:"country",code:"BDI"});
db.v.save({_key:"country-cambodia",name:"Cambodia",type:"country",code:"KHM"});
db.v.save({_key:"country-cameroon",name:"Cameroon",type:"country",code:"CMR"});
db.v.save({_key:"country-canada",name:"Canada",type:"country",code:"CAN"});
db.v.save({_key:"country-chad",name:"Chad",type:"country",code:"TCD"});
db.v.save({_key:"country-chile",name:"Chile",type:"country",code:"CHL"});
db.v.save({_key:"country-colombia",name:"Colombia",type:"country",code:"COL"});
db.v.save({_key:"country-cote-d-ivoire",name:"Coted'Ivoire",type:"country",code:"CIV"});
db.v.save({_key:"country-croatia",name:"Croatia",type:"country",code:"HRV"});
db.v.save({_key:"country-czech-republic",name:"CzechRepublic",type:"country",code:"CZE"});
db.v.save({_key:"country-denmark",name:"Denmark",type:"country",code:"DNK"});
db.v.save({_key:"country-ecuador",name:"Ecuador",type:"country",code:"ECU"});
db.v.save({_key:"country-egypt",name:"Egypt",type:"country",code:"EGY"});
db.v.save({_key:"country-eritrea",name:"Eritrea",type:"country",code:"ERI"});
db.v.save({_key:"country-finland",name:"Finland",type:"country",code:"FIN"});
db.v.save({_key:"country-france",name:"France",type:"country",code:"FRA"});
db.v.save({_key:"country-germany",name:"Germany",type:"country",code:"DEU"});
db.v.save({_key:"country-people-s-republic-of-china",name:"People'sRepublicofChina",type:"country",code:"CHN"});
//vertices:capitals
db.v.save({_key:"capital-algiers",name:"Algiers",type:"capital"});
db.v.save({_key:"capital-andorra-la-vella",name:"AndorralaVella",type:"capital"});
db.v.save({_key:"capital-asmara",name:"Asmara",type:"capital"});
db.v.save({_key:"capital-bandar-seri-begawan",name:"BandarSeriBegawan",type:"capital"});
db.v.save({_key:"capital-beijing",name:"Beijing",type:"capital"});
db.v.save({_key:"capital-berlin",name:"Berlin",type:"capital"});
db.v.save({_key:"capital-bogota",name:"Bogota",type:"capital"});
db.v.save({_key:"capital-brasilia",name:"Brasilia",type:"capital"});
164
ExampleData
db.v.save({_key:"capital-bridgetown",name:"Bridgetown",type:"capital"});
db.v.save({_key:"capital-brussels",name:"Brussels",type:"capital"});
db.v.save({_key:"capital-buenos-aires",name:"BuenosAires",type:"capital"});
db.v.save({_key:"capital-bujumbura",name:"Bujumbura",type:"capital"});
db.v.save({_key:"capital-cairo",name:"Cairo",type:"capital"});
db.v.save({_key:"capital-canberra",name:"Canberra",type:"capital"});
db.v.save({_key:"capital-copenhagen",name:"Copenhagen",type:"capital"});
db.v.save({_key:"capital-dhaka",name:"Dhaka",type:"capital"});
db.v.save({_key:"capital-gaborone",name:"Gaborone",type:"capital"});
db.v.save({_key:"capital-helsinki",name:"Helsinki",type:"capital"});
db.v.save({_key:"capital-kabul",name:"Kabul",type:"capital"});
db.v.save({_key:"capital-la-paz",name:"LaPaz",type:"capital"});
db.v.save({_key:"capital-luanda",name:"Luanda",type:"capital"});
db.v.save({_key:"capital-manama",name:"Manama",type:"capital"});
db.v.save({_key:"capital-nassau",name:"Nassau",type:"capital"});
db.v.save({_key:"capital-n-djamena",name:"N'Djamena",type:"capital"});
db.v.save({_key:"capital-ottawa",name:"Ottawa",type:"capital"});
db.v.save({_key:"capital-ouagadougou",name:"Ouagadougou",type:"capital"});
db.v.save({_key:"capital-paris",name:"Paris",type:"capital"});
db.v.save({_key:"capital-phnom-penh",name:"PhnomPenh",type:"capital"});
db.v.save({_key:"capital-prague",name:"Prague",type:"capital"});
db.v.save({_key:"capital-quito",name:"Quito",type:"capital"});
db.v.save({_key:"capital-saint-john-s",name:"SaintJohn's",type:"capital"});
db.v.save({_key:"capital-santiago",name:"Santiago",type:"capital"});
db.v.save({_key:"capital-sarajevo",name:"Sarajevo",type:"capital"});
db.v.save({_key:"capital-sofia",name:"Sofia",type:"capital"});
db.v.save({_key:"capital-thimphu",name:"Thimphu",type:"capital"});
db.v.save({_key:"capital-tirana",name:"Tirana",type:"capital"});
db.v.save({_key:"capital-vienna",name:"Vienna",type:"capital"});
db.v.save({_key:"capital-yamoussoukro",name:"Yamoussoukro",type:"capital"});
db.v.save({_key:"capital-yaounde",name:"Yaounde",type:"capital"});
db.v.save({_key:"capital-zagreb",name:"Zagreb",type:"capital"});
//edges:continent->world
db.e.save("v/continent-africa","v/world",{type:"is-in"});
db.e.save("v/continent-asia","v/world",{type:"is-in"});
db.e.save("v/continent-australia","v/world",{type:"is-in"});
db.e.save("v/continent-europe","v/world",{type:"is-in"});
db.e.save("v/continent-north-america","v/world",{type:"is-in"});
db.e.save("v/continent-south-america","v/world",{type:"is-in"});
//edges:country->continent
db.e.save("v/country-afghanistan","v/continent-asia",{type:"is-in"});
db.e.save("v/country-albania","v/continent-europe",{type:"is-in"});
db.e.save("v/country-algeria","v/continent-africa",{type:"is-in"});
db.e.save("v/country-andorra","v/continent-europe",{type:"is-in"});
db.e.save("v/country-angola","v/continent-africa",{type:"is-in"});
db.e.save("v/country-antigua-and-barbuda","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-argentina","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-australia","v/continent-australia",{type:"is-in"});
db.e.save("v/country-austria","v/continent-europe",{type:"is-in"});
db.e.save("v/country-bahamas","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-bahrain","v/continent-asia",{type:"is-in"});
db.e.save("v/country-bangladesh","v/continent-asia",{type:"is-in"});
db.e.save("v/country-barbados","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-belgium","v/continent-europe",{type:"is-in"});
db.e.save("v/country-bhutan","v/continent-asia",{type:"is-in"});
db.e.save("v/country-bolivia","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-bosnia-and-herzegovina","v/continent-europe",{type:"is-in"});
db.e.save("v/country-botswana","v/continent-africa",{type:"is-in"});
db.e.save("v/country-brazil","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-brunei","v/continent-asia",{type:"is-in"});
db.e.save("v/country-bulgaria","v/continent-europe",{type:"is-in"});
db.e.save("v/country-burkina-faso","v/continent-africa",{type:"is-in"});
db.e.save("v/country-burundi","v/continent-africa",{type:"is-in"});
db.e.save("v/country-cambodia","v/continent-asia",{type:"is-in"});
db.e.save("v/country-cameroon","v/continent-africa",{type:"is-in"});
db.e.save("v/country-canada","v/continent-north-america",{type:"is-in"});
db.e.save("v/country-chad","v/continent-africa",{type:"is-in"});
db.e.save("v/country-chile","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-colombia","v/continent-south-america",{type:"is-in"});
db.e.save("v/country-cote-d-ivoire","v/continent-africa",{type:"is-in"});
db.e.save("v/country-croatia","v/continent-europe",{type:"is-in"});
db.e.save("v/country-czech-republic","v/continent-europe",{type:"is-in"});
db.e.save("v/country-denmark","v/continent-europe",{type:"is-in"});
db.e.save("v/country-ecuador","v/continent-south-america",{type:"is-in"});
165
ExampleData
db.e.save("v/country-egypt","v/continent-africa",{type:"is-in"});
db.e.save("v/country-eritrea","v/continent-africa",{type:"is-in"});
db.e.save("v/country-finland","v/continent-europe",{type:"is-in"});
db.e.save("v/country-france","v/continent-europe",{type:"is-in"});
db.e.save("v/country-germany","v/continent-europe",{type:"is-in"});
db.e.save("v/country-people-s-republic-of-china","v/continent-asia",{type:"is-in"});
//edges:capital->country
db.e.save("v/capital-algiers","v/country-algeria",{type:"is-in"});
db.e.save("v/capital-andorra-la-vella","v/country-andorra",{type:"is-in"});
db.e.save("v/capital-asmara","v/country-eritrea",{type:"is-in"});
db.e.save("v/capital-bandar-seri-begawan","v/country-brunei",{type:"is-in"});
db.e.save("v/capital-beijing","v/country-people-s-republic-of-china",{type:"is-in"});
db.e.save("v/capital-berlin","v/country-germany",{type:"is-in"});
db.e.save("v/capital-bogota","v/country-colombia",{type:"is-in"});
db.e.save("v/capital-brasilia","v/country-brazil",{type:"is-in"});
db.e.save("v/capital-bridgetown","v/country-barbados",{type:"is-in"});
db.e.save("v/capital-brussels","v/country-belgium",{type:"is-in"});
db.e.save("v/capital-buenos-aires","v/country-argentina",{type:"is-in"});
db.e.save("v/capital-bujumbura","v/country-burundi",{type:"is-in"});
db.e.save("v/capital-cairo","v/country-egypt",{type:"is-in"});
db.e.save("v/capital-canberra","v/country-australia",{type:"is-in"});
db.e.save("v/capital-copenhagen","v/country-denmark",{type:"is-in"});
db.e.save("v/capital-dhaka","v/country-bangladesh",{type:"is-in"});
db.e.save("v/capital-gaborone","v/country-botswana",{type:"is-in"});
db.e.save("v/capital-helsinki","v/country-finland",{type:"is-in"});
db.e.save("v/capital-kabul","v/country-afghanistan",{type:"is-in"});
db.e.save("v/capital-la-paz","v/country-bolivia",{type:"is-in"});
db.e.save("v/capital-luanda","v/country-angola",{type:"is-in"});
db.e.save("v/capital-manama","v/country-bahrain",{type:"is-in"});
db.e.save("v/capital-nassau","v/country-bahamas",{type:"is-in"});
db.e.save("v/capital-n-djamena","v/country-chad",{type:"is-in"});
db.e.save("v/capital-ottawa","v/country-canada",{type:"is-in"});
db.e.save("v/capital-ouagadougou","v/country-burkina-faso",{type:"is-in"});
db.e.save("v/capital-paris","v/country-france",{type:"is-in"});
db.e.save("v/capital-phnom-penh","v/country-cambodia",{type:"is-in"});
db.e.save("v/capital-prague","v/country-czech-republic",{type:"is-in"});
db.e.save("v/capital-quito","v/country-ecuador",{type:"is-in"});
db.e.save("v/capital-saint-john-s","v/country-antigua-and-barbuda",{type:"is-in"});
db.e.save("v/capital-santiago","v/country-chile",{type:"is-in"});
db.e.save("v/capital-sarajevo","v/country-bosnia-and-herzegovina",{type:"is-in"});
db.e.save("v/capital-sofia","v/country-bulgaria",{type:"is-in"});
db.e.save("v/capital-thimphu","v/country-bhutan",{type:"is-in"});
db.e.save("v/capital-tirana","v/country-albania",{type:"is-in"});
db.e.save("v/capital-vienna","v/country-austria",{type:"is-in"});
db.e.save("v/capital-yamoussoukro","v/country-cote-d-ivoire",{type:"is-in"});
db.e.save("v/capital-yaounde","v/country-cameroon",{type:"is-in"});
db.e.save("v/capital-zagreb","v/country-croatia",{type:"is-in"});
166
WorkingwithEdges
Edges,Identifiers,Handles
ThisisanintroductiontoArangoDB'sinterfaceforedges.Edgesmaybeusedingraphs.HereweworkwithedgesfromtheJavaScript
shellarangosh.ForotherlanguagesseethecorrespondinglanguageAPI.
Agraphdatamodelalwaysconsistsofatleasttwocollections:therelationsbetweenthenodesinthegraphsarestoredinan"edges
collection",thenodesinthegrapharestoredindocumentsinregularcollections.
EdgesinArangoDBarespecialdocuments.Inadditiontothesystemattributes_key,_idand_rev,theyhavetheattributes_fromand
_to,whichcontaindocumenthandles,namelythestart-pointandtheend-pointoftheedge.
Example:
the"edge"collectionstorestheinformationthatacompany'sreceptionissub-unittotheservicesunitandtheservicesunitissubunittotheCEO.Youwouldexpressthisrelationshipwiththe_fromand_toattributes
the"normal"collectionstoresallthepropertiesaboutthereception,e.g.that20peopleareworkingthereandtheroomnumberetc
_fromisthedocumenthandleofthelinkedvertex(incomingrelation)
_toisthedocumenthandleofthelinkedvertex(outgoingrelation)
Edgecollectionsarespecialcollectionsthatstoreedgedocuments.Edgedocumentsareconnectiondocumentsthatreferenceother
documents.Thetypeofacollectionmustbespecifiedwhenacollectioniscreatedandcannotbechangedafterwards.
Tochangeedgeendpointsyouwouldneedtoremoveolddocument/edgeandinsertnewone.Otherfieldscanbeupdatedasindefault
collection.
WorkingwithEdges
Edgesarenormaldocumentsthatalwayscontaina _fromanda _toattribute.
167
Pregel
DistributedIterativeGraphProcessing(Pregel)
Distributedgraphprocessingenablesyoutodoonlineanalyticalprocessingdirectlyongraphsstoredintoarangodb.Thisisintendedto
helpyougainanalyticalinsightsonyourdata,withouthavingtouseexternalprocessingsytems.Examplesofalgorithmstoexecuteare
PageRank,VertexCentrality,VertexCloseness,ConnectedComponents,CommunityDetection.Thissystemisnotusefulfortypical
onlinequeries,whereyoujustdoworkonasmallsetofvertices.ThesekindoftasksarebettersuitedforAQL.
TheprocessingsysteminsideArangoDBisbasedon:Pregel:ASystemforLarge-ScaleGraphProcessing–M alewiczetal.(Google)
2010Thisconceptenablesustoperformdistributedgraphprocessing,withouttheneedfordistributedgloballocking.
Prerequisites
IfyouarerunningasingleArangoDBinstanceinsingle-servermode,therearenorequirementsregardingthemodelingofyourdata.All
youneedisatleastonevertexcollectionandoneedgecollection.Notethattheperformancemaybebetter,ifthenumberofyourshards/
collectionsmatchesthenumberofCPUcores.
WhenyouuseArangoDBCommunityeditioninclustermode,youmightneedtomodelyourcollectionsinacertainwaytoensure
correctresults.Formoreinformationseethenextsection.
RequirementsforCollectionsinaCluster(NonSmartGraph)
Toenableiterativegraphprocessingforyourdata,youwillneedtoensurethatyourvertexandedgecollectionsareshardedinaspecific
way.
ThepregelcomputingmodelrequiresalledgestobepresentontheDBServerwherethevertexdocumentidentifiedbythe _fromvalue
islocated.Thismeansthevertexcollectionsneedtobeshardedby'_key'andtheedgecollectionwillneedtobeshardedafteranattribute
whichalwayscontainsthe'_key'ofthevertex.
Ourimplementationcurrentlyrequireseveryedgecollectiontobeshardedaftera"vertex"attributes,additionallyyouwillneedto
specifythekey distributeShardsLikeandanequalnumberofshardsoneverycollection.Onlyiftheserequirementsaremetcan
ArangoDBplacetheedgesandverticescorrectly.
Forexampleyoumightcreateyourcollectionslikethis:
//Createmainvertexcollection:
db._create("vertices",{
shardKeys:['_key'],
numberOfShards:8
});
//Optionallycreatearbitraryadditionalvertexcollections
db._create("additonal",{
distributeShardsLike:"vertices",
numberOfShards:8
});
//Create(oneormore)edge-collections:
db._createEdgeCollection("edges",{
shardKeys:['vertex'],
distributeShardsLike:"vertices",
numberOfShards:8
});
Youwillneedtoensurethatedgedocumentscontainthepropervaluesintheirshardingattribute.Foravertexdocumentwiththe
followingcontent {_key:"A",value:0}thecorrespondingedgedocumentswouldhavelooklikethis:
{_from:"vertices/A",_to:"vertices/B",vertex:"A"}
{_from:"vertices/A",_to:"vertices/C",vertex:"A"}
{_from:"vertices/A",_to:"vertices/D",vertex:"A"}
...
168
Pregel
ThiswillensurethatoutgoingedgedocumentswillbeplacedonthesameDBServerasthevertex.Withoutthecorrectplacementofthe
edges,thepregelgraphprocessingsystemwillnotworkcorrectly,becauseedgeswillnotloadcorrectly.
ArangoshAPI
StartinganAlgorithmExecution
ThepregelAPIisaccessiblethroughthe @arangodb/pregelpackage.Tostartanexecutionyouneedtospecifythealgorithmnameand
thevertexandedgecollections.Alternativelyyoucanspecifyanamedgraph.Additionallyyoucanspecifycustomparameterswhich
varyforeachalgorithm.The startmethodwillalwaysauniqueIDwhichcanbeusedtointeractwiththealgorithmandlateron.
Thebelowversionofthe startmethodcanbeusedfornamedgraphs:
varpregel=require("@arangodb/pregel");
varparams={};
varexecution=pregel.start("<algorithm>","<yourgraph>",params);
Paramsneedstobeanobject,thevalidkeysarementionedbelowinthesectionAlgorithms
Alternativelyyoumightwanttospecifythevertexandedgecollectionsdirectly.Thecall-syntaxofthe start``methodchangesinthis
case.ThesecondargumentmustbeanobjectwiththekeysvertexCollections andedgeCollections`.
varpregel=require("@arangodb/pregel");
varparams={};
varexecution=pregel.start("<algorithm>",{vertexCollections:["vertices"],edgeCollections:["edges"]},{});
Thelastargumentisstilltheparameterobject.Seebelowforalistofalgorithmsandparameters.
StatusofanAlgorithmExecution
Thecodereturnedbythe pregel.start(...)methodcanbeusedtotrackthestatusofyouralgorithm.
varexecution=pregel.start("sssp","demograph",{source:"vertices/V"});
varstatus=pregel.status(execution);
Theresultwilltellyouthecurrentstatusofthealgorithmexecution.Itwilltellyouthecurrent stateoftheexecution,thecurrent
globalsuperstep,theruntime,theglobalaggregatorvaluesaswellasthenumberofsendandreceivedmessages.
Validvaluesforthe statefieldinclude:
"running"algorithmisstillrunning
"done":Theexecutionisdone,theresultmightnotbewrittenbackintothecollectionyet.
"canceled":Theexecutionwaspermanentlycanceled,eitherbytheuserorbyanerror.
"inerror":Theexeuctionisinanerrorstate.ThiscanbecausedbyprimaryDBServersbeingnotreachableorbeingnonresponsive.
Theexecutionmightrecoverlater,orswitchto"canceled"ifitwasnotabletorecoversuccessfuly
"recovering":Theexecutionisactivelyrecovering,willswitchbackto"running"iftherecoverywassuccessful
Theobjectreturnedbythe statusmethodmightforexamplelooksomethinglikethis:
{
"state":"running",
"gss":12,
"totalRuntime":123.23,
"aggregators":{
"converged":false,
"max":true,
"phase":2
},
"sendCount":3240364978,
"receivedCount":3240364975
}
169
Pregel
CancelinganExecution/Discardingresults
Tocancelanexecutionwhichisstillrunnning,anddiscardanyintermediareresultsyoucanusethe cancelmethod.Thiswill
immediatlyfreeallmemorytakenupbytheexecution,andwillmakeyouloseallintermediarydata.
Youmightgetinconsistentresultsifyoucancelanexecutionwhileitisalreadyinit's donestate.Thedataiswrittenmulti-threadedinto
allcollectionshardsatonce,thismeanstherearemultipletransactionssimultaniously.Atransactionmightalreadybecommitedwhen
youcanceltheexecutionjob,thereforeyoumightseetheresultinyourcollection.Thisdoesnotapplyifyouconfiguredtheexecutionto
notwritedataintothecollection.
//startasinglesourceshortestpathjob
varexecution=pregel.start("sssp","demograph",{source:"vertices/V"});
pregel.cancel(execution);
AQLintegration
ArangoDBsupportsretrievingtemporarypregelresultsthroughtheArangoDBquerylanguage(AQL).Whenourgraphprocessing
subsystemfinishesexecutinganalgorithm,theresultcaneitherbewrittenbackintothedatabaseorkeptinmemory.Inbothcasesthe
resultcanbequeriedviaAQL.Ifthedatawasnotwrittentothedatabasestoreitisonlyheldtemporarily,untiltheusercallsthe
cancelmethodForexampleausermightwanttoqueryonlynodeswiththemostrankfromtheresultsetofaPageRankexecution.
FORvINPREGEL_RESULT(<handle>)
FILTERv.value>=0.01
RETURNv._key
AvailableAlgorithms
Thereareanumberofgeneralparameterswhichapplytoalmostallalgorithms:
store:Isperdefaulttrue,thepregelenginewillwriteresultsbacktothedatabase.ifthevalueisfalsethenyoucanquerythe
resultsviaAQLk,seeAQLintegration.
maxGSS:M aximumnumberofglobaliterationsforthisalgorithm
parallelism:Numberofparellelthreadstouseperworker.Doesnotinfluencethenumberofthreadsusedtoload
orstoredatafromthedatabase(thisdependsonthenumberofshards).
async:Algorithmswichsupportasyncmode,willrunwithoutsynchronizedglobaliterations,mightleadtoperformanceincreases
ifyouhaveloadimbalances.
resultField:M ostalgorithmswillwritetheresultintothisfield
PageRank
PageRankisawellknownalgorithmtorankdocumentsinagraph.Thealgorithmwillrununtiltheexecutionconverges.Specifyacustom
thresholdwiththeparameter threshold,torunforafixednumberofiterationsusethe maxGSSparameter.
varpregel=require("@arangodb/pregel");
pregel.start("pagerank","graphname",{maxGSS:100,threshold:0.00000001})
Single-SourceShortestPath
Calculatesthedistanceofeachvertextoacertainshortestpath.Thealgorithmwillrununtilitconverges,theiterationsareboundbythe
diameter(thelongestshortestpath)ofyourgraph.
varpregel=require("@arangodb/pregel");
pregel.start("sssp","graphname",{source:"vertices/1337"})
ConnectedComponents
170
Pregel
Therearetwoalgorithmstofindconnectedcomponentsinagraph.Tofindweaklyconnectedcomponents(WCC)youcanusethe
algorithmnamed"connectedcomponents",tofindstronglyconnectedcomponents(SCC)youcanusethealgorithmnamed"scc".Both
algorithmwillassignacomponentIDtoeachvertex.
Aweaklyconnectedcomponentsmeansthatthereexistapathfromeveryvertexpairinthatcomponent.WCCisaverysimpleandfast
algorithm,whichwillonlyworkcorrectlyonundirectedgraphs.Yourresultsondirectedgraphsmayvary,dependingonhowconnected
yourcomponentsare.
InthecaseofSCCacomponentmeanseveryvertexisreachablefromanyothervertexinthesamecomponent.Thealgorithmismore
complexthantheWCCalgorithmandrequiresmoreRAM ,becauseeachvertexneedstostoremuchmorestate.ConsiderusingWCCif
youthinkyourdatamaybesuitableforit.
varpregel=require("@arangodb/pregel");
//weaklyconnectedcomponents
pregel.start("connectedcomponents","graphname")
//stronglyconnectedcomponents
pregel.start("scc","graphname")
Hyperlink-InducedTopicSearch(HITS)
HITSisalinkanalysisalgorithmthatratesWebpages,developedbyJonKleinberg(Thealgorithmisalsoknownashubsand
authorities).
TheideabehindHubsandAuthoritiescomesfromthetypicalstructureoftheweb:Certainwebsitesknownashubs,serveaslarge
directoriesthatarenotactuallyauthoritativeontheinformationthattheyhold.Thesehubsareusedascompilationsofabroadcatalogof
informationthatleadsusersdirecttootherauthoritativewebpages.Thealgorithmassignseachvertextwoscores:Theauthority-score
andthehub-score.Theauthorityscorerateshowmanygoodhubspointtoaparticularvertex(orwebpage),thehubscorerateshow
good(authoritative)theverticespointedtoare.Formoreseehttps://en.wikipedia.org/wiki/HITS_algorithm
Ourversionofthealgorithmconvergesafteracertainamountoftime.Theparameterthresholdcanbeusedtosetalimitforthe
convergence(measuredasmaximumabsolutedifferenceofthehubandauthorityscoresbetweenthecurrentandlastiteration)Whenyou
specifytheresultfieldname,thehubscorewillbestoredin"_hub"andtheauthorityscorein"_auth".Thealgorithmcanbeexecutedlike
this:
varpregel=require("@arangodb/pregel");
varhandle=pregel.start("hits","yourgraph",{threshold:0.00001,resultField:"score"});
VertexCentrality
Centralitymeasureshelpidentifythemostimportantverticesinagraph.Theycanbeusedinawiderangeofapplications:Forexample
theycanbeusedtoidentifyinfluencersinsocialnetworks,ormiddle-meninterroristnetworks.Therearevariousdefinitionsfor
centrality,thesimplestonebeingthevertexdegree.Thesedefinitionswerenotdesignedwithscalabilityinmind.Itisprobably
impossibletodiscoveranefficientalgorithmwhichcomputestheminadistributedway.Fortunatelytherearescalablesubstitutions
available,whichshouldbeequallyusableformostusecases.
171
Pregel
EffectiveCloseness
Acommondefinitionsofcentralityistheclosenesscentrality(orcloseness).Theclosenessofavertexinagraphistheinverseaverage
lengthoftheshortestpathbetweenthevertexandallothervertices.Forverticesx,yandshortestdistanced(y,x)itisdefinedas
EffectiveClosenessapproximatestheclosenessmeasure.Thealgorithmworksbyiterativelyestimatingthenumberofshortestpaths
passingthrougheachvertex.Thescorewillapproximatesthetherealclosenessscore,sinceitisnotpossibletoactuallycountallshortest
pathsduetothehorrendousO(n^2d)memoryrequirements.ThealgorithmisfromthepaperCentralitiesinLargeNetworks:Algorithms
andObservations(UKanget.al.2011)*
ArangoDBsimplementationapproximatesthenumberofshortestpathineachiterationbyusingaHyperLogLogcounterwith64
buckets.Thisshouldworkwellonlargegraphsandonsmalleronesaswell.ThememoryrequirementsshouldbeO(n*d)wherenisthe
numberofverticesanddthediameterofyourgraph.Eachvertexwillstoreacounterforeachiterationofthealgorithm.Thealgorithmcan
beusedlikethis
varpregel=require("@arangodb/pregel");
varhandle=pregel.start("effectivecloseness","yourgraph",{resultField:"closeness"});
LineRank
Anothercommonmeasureisthebetweenness*centrality:Itmeasuresthenumberoftimesavertexispartofshortestpathsbetweenany
pairsofvertices.Foravertexvbetweennessisdefinedas
Wheretheσrepresentsthenumberofshortestpathsbetweenxandy,andσ(v)representsthenumberofpathsalsopassingthrougha
vertexv.Byintuitionavertexwithhigherbetweenesscentralitywillhavemoreinformationpassingthroughit.
LineRankapproximatestherandomwalkbetweennessofeveryvertexinagraph.Thisistheprobabilitythatsomeonestartingonan
arbitaryvertex,willvisitthisnodewhenherandomlychoosesedgestovisit.Thealgoruthmessentiallybuildsalinegraphoutofyour
graph(switchestheverticesandedges),andthencomputesascoresimilartoPageRank.Thiscanbeconsideredascalableequivalentto
vertexbetweeness,whichcanbeexecuteddistributedlyinArangoDB.ThealgorithmisfromthepaperCentralitiesinLargeNetworks:
AlgorithmsandObservations(UKanget.al.2011)
172
Pregel
varpregel=require("@arangodb/pregel");
varhandle=pregel.start("linerank","yourgraph",{"resultField":"rank"});
CommunityDetection
Graphsbasedonrealworldnetworksoftenhaveacommunitystructure.Thismeansitispossibletofindgroupsofverticessuchthat
eacheachvertexgroupisinternallymoredenselyconnectedthanoutsidethegroup.Thishasmanyapplicationswhenyouwantto
analyzeyournetworks,forexampleSocialnetworksincludecommunitygroups(theoriginoftheterm,infact)basedoncommon
location,interests,occupation,etc.
LabelPropagation
LabelPropagationcanbeusedtoimplementcommunitydetectiononlargegraphs.Theideaisthateachvertexshouldbeinthe
communitythatmostofhisneighboursarein.WeiterativelydeteminethisbyfirstassigningrandomCommunityID's.Theneach
itertation,avertexwillsendit'scurrentcommunityIDtoallhisneighborvertices.TheneachvertexadoptsthecommunityIDhereceived
mostfrequentlyduringtheiteration.
Thealgorithmrunsuntilitconverges,whichlikelyneverreallyhappensonlargegraphs.Thereforeyouneedtospecifyamaximum
iterationboundwhichsuitsyou.Thedefaultboundis500iterations,whichislikelytoolargeforyourapplication.Shouldworkbeston
undirectedgraphs,resultsondirectedgraphsmightvarydependingonthedensityofyourgraph.
varpregel=require("@arangodb/pregel");
varhandle=pregel.start("labelpropagation","yourgraph",{maxGSS:100,resultField:"community"});
S peaker-ListenerLabelPropagation
173
FoxxM icroservices
Foxx
Traditionally,server-sideprojectshavebeendevelopedasstandaloneapplicationsthatguidethecommunicationbetweentheclient-side
frontendandthedatabasebackend.Thishasledtoapplicationsthatwereeitherdevelopedassinglemonolithsorthatduplicateddata
accessanddomainlogicacrossallservicesthathadtoaccessthedatabase.Additionally,toolstoabstractawaytheunderlyingdatabase
callscouldincuralotofnetworkoverheadwhenusingremotedatabaseswithoutcarefuloptimization.
ArangoDBallowsapplicationdeveloperstowritetheirdataaccessanddomainlogicasmicroservicesrunningdirectlywithinthedatabase
withnativeaccesstoin-memorydata.TheFoxxmicroserviceframeworkmakesiteasytoextendArangoDB'sownRESTAPIwith
customHTTPendpointsusingmodernJavaScriptrunningonthesameV8engineyouknowfromNode.jsandtheGoogleChromeweb
browser.
Unliketraditionalapproachestostoringlogicinthedatabase(likestoredprocedures),thesemicroservicescanbewrittenasregular
structuredJavaScriptapplicationsthatcanbeeasilydistributedandversioncontrolled.Dependingonyourproject'sneedsFoxxcanbe
usedtobuildanythingfromoptimizedRESTendpointsperformingcomplexdataaccesstoentirestandaloneapplicationsrunning
directlyinsidethedatabase.
174
Ataglance
Foxxataglance
EachFoxxserviceisdefinedbyaJSONmanifestspecifyingtheentrypoint,anyscriptsdefinedbytheservice,possibleconfiguration
optionsandFoxxdependencies,aswellasothermetadata.Withinaservice,theseoptionsareexposedastheservicecontext.
AttheheartoftheFoxxframeworkliestheFoxxRouterwhichisusedtodefineHTTPendpoints.Aservicecanaccessthedatabase
eitherdirectlyfromitscontextusingprefixedcollectionsortheArangoDBdatabaseAPI.
WhileFoxxisprimarilydesignedtobeusedtoaccessthedatabaseitself,ArangoDBalsoprovidesanAPItomakeHTTPrequeststo
externalservices.
Finally,scriptscanbeusedtoperformone-offtasks,whichcanalsobescheduledtobeperformedasynchronouslyusingthebuilt-injob
queue.
Howdoesitwork
FoxxservicesconsistofJavaScriptcoderunningintheV8JavaScriptruntimeembeddedinsideArangoDB.Eachserviceismountedin
eachavailableV8context(thenumberofcontextscanbeadjustedintheArangoDBconfiguration).Incomingrequestsaredistributed
accrossthesecontextsautomatically.
Ifyou'recomingfromanotherJavaScriptenvironmentlikeNode.jsthisissimilartorunningmultipleNode.jsprocessesbehindaload
balancer:youshouldnotrelyonserver-sidestate(otherthanthedatabaseitself)betweendifferentrequestsasthereisnowayofmaking
sureconsecutiverequestswillbehandledinthesamecontext.
BecausetheJavaScriptcodeisrunninginsidethedatabaseanotherdifferenceisthatallFoxxandArangoDBAPIsarepurelysynchronous
andshouldbeconsideredblocking.Thisisespeciallyimportantfortransactions,whichinArangoDBcanexecutearbitrarycodebutmay
havetolockentirecollections(effectivelypreventinganydatatobewritten)untilthecodehascompleted.
Forinformationonhowthisaffectsinteroperabilitywiththird-partyJavaScriptmoduleswrittenforotherJavaScriptenvironmentssee
thechapterondependencies.
Developmentmode
Developmentmodeallowsyoutomakechangestodeployedservicesin-placedirectlyonthedatabaseserver'sfilesystemwithout
downloadingandre-uploadingtheservicebundle.Additionallyerrormessageswillcontainstacktraces.
YoucantoggledevelopmentmodeonandoffintheservicesettingstabofthewebinterfaceorusingtheHTTPAPI.Onceactivatedthe
service'sfilesystempathwillbeshownintheinfotab.
Onceenabledtheservice'ssourcefilesandmanifestwillbere-evaluated,andthesetupscript(ifpresent)re-executed,everytimearoute
oftheserviceisaccessed,effectivelyre-deployingtheserviceoneveryrequest.Asthenameindicatesthisisintendedtobeusedstrictly
duringdevelopmentandismostdefinitelyabadideaonproductionservers.Theadditionalinformationexposedduringdevelopment
modemayincludefilesystempathsandpartsoftheservice'ssourcecode.
Alsonotethatifyouareservingstaticfilesaspartofyourservice,accessingthesefilesfromabrowsermayalsotriggerare-deployment
oftheservice.Finally,makingHTTPrequeststoaservicerunningindevelopmentmodefromwithintheservice(i.e.usingthe
@arangodb/requestmoduletoaccesstheserviceitself)isprobablynotagoodideaeither.
Bewareofdeletingthedatabasetheserviceisdeployedon:itwillerasethesourcefilesoftheservicealongwiththecollections.You
shouldbackupthecodeyouworkedonindevelopmentbeforedoingthattoavoidlosingyourprogress.
Foxxstore
TheFoxxstoreprovidesaccesstoanumberofready-to-useofficialandcommunity-maintainedFoxxservicesyoucaninstallwithasingle
click,includingexampleservicesandwrappersforexternalSaaStoolsliketransactionale-mailservices,bugloggersoranalyticstrackers.
YoucanfindtheFoxxstoreinthewebinterfacebyusingtheAddServicebuttonintheservicelist.
175
Ataglance
Cluster-Foxx
WhenrunningArangoDBinaclustertheFoxxserviceswillrunoneachcoordinator.Installing,upgradinganduninstallingservicesonany
coordinatorwillautomaticallydistributetheservicetotheothercoordinators,makingdeploymentsaseasyasinsingle-servermode.
However,thismeanstherearesomelimitations:
Youshouldavoidanykindoffilesystemstatebeyondthedeployedservicebundleitself.Don'twritedatatothefilesystemorencode
anyexpectationsofthefilesystemstateotherthanthefilesintheservicefolderthatwereinstalledaspartoftheservice(e.g.fileuploads
orcustomlogfiles).
Additionally,thedevelopmentmodewillleadtoaninconsistentstateoftheclusteruntilitisdisabled.Whileaserviceisrunningin
developmentmodeyoucanmakechangestotheserviceonthefilesystemofanycoordinatorandseethemreflectedinrealtimejustlike
whenrunningArangoDBasasingleserver.Howeverthechangesmadeononecoordinatorwillnotbereflectedacrossothercoordinators
untilthedevelopmentmodeisdisabled.Whendisablingthedevelopmentmodeforaservice,thecoordinatorwillcreateanewbundleand
distributeitacrosstheservicelikeamanualupgradeoftheservice.
Forthesereasonsitisstronglyrecommendednottousedevelopmentmodeinaclusterwithmultiplecoordinatorsunlessyouaresure
thatnorequestsorchangeswillbemadetoothercoordinatorswhileyouaremodifyingtheservice.Usingdevelopmentmodeina
productionclusterisextremelyunsafeandhighlydiscouraged.
176
Gettingstarted
GettingStarted
We'regoingtostartwithanemptyfolder.Thiswillbetherootfolderofourservices.Youcannameitsomethingcleverbutforthecourse
ofthisguidewe'llassumeit'scalledthenameofyourservice: getting-started.
Firstweneedtocreateamanifest.Createanewfilecalled manifest.jsonandaddthefollowingcontent:
{
"engines":{
"arangodb":"^3.0.0"
}
}
ThisjusttellsArangoDBtheserviceiscompatiblewithversions3.0.0andlater(allthewayuptobutnotincluding4.0.0),allowingolder
versionsofArangoDBtounderstandthatthisservicelikelywon'tworkforthemandnewerversionswhatbehaviortoemulateshould
theystillsupportit.
Thelittlehattotheleftoftheversionnumberisnotatypo,it'scalleda"caret"andindicatestheversionrange.Foxxusessemantic
versioning(alsocalled"semver")formostofitsversionhandling.Youcanfindoutmoreabouthowsemverworksattheofficialsemver
website.
Nextwe'llneedtospecifyanentrypointtoourservice.ThisistheJavaScriptfilethatwillbeexecutedtodefineourservice'sHTTP
endpoints.Wecandothisbyaddinga"main"fieldtoourmanifest:
{
"engines":{
"arangodb":"^3.0.0"
},
"main":"index.js"
}
That'sallweneedinourmanifestfornow,solet'snextcreatethe index.jsfile:
'usestrict';
constcreateRouter=require('@arangodb/foxx/router');
constrouter=createRouter();
module.context.use(router);
Thefirstlinecausesourfiletobeinterpretedusingstrictmode.AllexamplesintheArangoDBdocumentationassumestrictmode,so
youmightwanttofamiliarizeyourselfwithitifyouhaven'tencountereditbefore.
Thesecondlineimportsthe @arangodb/foxx/routermodulewhichprovidesafunctionforcreatingnewFoxxrouters.We'reusingthis
functiontocreateanew routerobjectwhichwe'llbeusingforourservice.
The module.contextistheso-calledFoxxcontextorservicecontext.ThisvariableisavailableinallfilesthatarepartofyourFoxx
serviceandprovidesaccesstoFoxxAPIsspecifictothecurrentservice,likethe usemethod,whichtellsFoxxtomountthe routerin
thisservice(andtoexposeitsroutestoHTTP).
Nextlet'sdefinearoutethatprintsagenericgreeting:
//continued
router.get('/hello-world',function(req,res){
res.send('HelloWorld!');
})
.response(['text/plain'],'Agenericgreeting.')
.summary('Genericgreeting')
.description('Printsagenericgreeting.');
The routerprovidesthemethods get, post,etccorrespondingtoeachHTTPverbaswellasthecatch-all all.Thesemethods
indicatethatthegivenrouteshouldbeusedtohandleincomingrequestswiththegivenHTTPverb(oranymethodwhenusing all).
177
Gettingstarted
Thesemethodstakeanoptionalpath(ifomitted,itdefaultsto "/")aswellasarequesthandler,whichisafunctiontakingthe req
(request)and res(response)objectstohandletheincomingrequestandgeneratetheoutgoingresponse.Ifyouhaveusedtheexpress
frameworkinNode.js,youmayalreadybefamiliarwithhowthisworks,otherwisecheckoutthechapteronroutes.
Theobjectreturnedbytherouter'smethodsprovidesadditionalmethodstoattachmetadataandvalidationtotheroute.We'reusing
summaryand descriptiontodocumentwhattheroutedoes--thesearen'tstrictlynecessarybutgiveussomeniceauto-generated
documentation.The responsemethodletsusadditionallydocumenttheresponsecontenttypeandwhattheresponsebodywill
represent.
Tryitout
AtthispointyoucanuploadtheservicefolderasaziparchivefromthewebinterfaceusingtheServicestab.
ClickAddServicethenpicktheZipoptioninthedialog.Youwillneedtoprovideamountpath,whichistheURLprefixatwhichthe
servicewillbemounted(e.g. /getting-started).
Onceyouhavepickedtheziparchiveusingthefilepicker,theuploadshouldbeginimmediatelyandyourserviceshouldbeinstalled.
OtherwisepresstheInstallbuttonandwaitforthedialogtodisappearandtheservicetoshowupintheservicelist.
Clickanywhereonthecardwithyourmountpathonthelabeltoopentheservice'sdetails.
IntheAPIdocumentationyoushouldseetheroutewedefinedearlier( /hello-world)withtheword GETnexttoitindicatingthe
HTTPmethoditsupportsandthe summaryweprovidedontheright.Byclickingontheroute'spathyoucanopenthedocumentation
fortheroute.
Notethatthe descriptionweprovidedappearsinthegenerateddocumentationaswellasthedescriptionweaddedtothe response
(whichshouldcorrectlyindicatethecontenttype text/plain,i.e.plaintext).
ClicktheTryitout!buttontosendarequesttotherouteandyoushouldseeanexamplerequestwiththeservice'sresponse:"Hello
World!".
Congratulations!Youhavejustcreated,installedandusedyourfirstFoxxservice.
Parametervalidation
Let'saddanotherroutethatprovidesamorepersonalizedgreeting:
//continued
constjoi=require('joi');
router.get('/hello/:name',function(req,res){
res.send(`Hello${req.pathParams.name}`);
})
.pathParam('name',joi.string().required(),'Nametogreet.')
.response(['text/plain'],'Apersonalizedgreeting.')
.summary('Personalizedgreeting')
.description('Printsapersonalizedgreeting.');
Thefirstlineimportsthe joimodulefromnpmwhichcomesbundledwithArangoDB.Joiisavalidationlibrarythatisused
throughoutFoxxtodefineschemasandparametertypes.
Note:Youcanbundleyourownmodulesfromnpmbyinstallingtheminyourservicefolderandmakingsurethe node_modulesfolderis
includedinyourziparchive.Formoreinformationseethesectiononmoduledependenciesinthechapterondependencies.
The pathParammethodallowsustospecifyparametersweareexpectinginthepath.Thefirstargumentcorrespondstotheparameter
nameinthepath,thesecondargumentisajoischematheparameterisexpectedtomatchandthefinalargumentservestodescribethe
parameterintheAPIdocumentation.
Thepathparametersareaccessiblefromthe pathParamspropertyoftherequestobject.We'reusingatemplatestringtogeneratethe
server'sresponsecontainingtheparameter'svalue.
178
Gettingstarted
NotethatrouteswithpathparametersthatfailtovalidatefortherequestURLwillbeskippedasiftheywouldn'texist.Thisallowsyou
todefinemultipleroutesthatareonlydistinguishedbytheschemasoftheirpathparameters(e.g.aroutetakingonlynumericparameters
andonetakinganystringasafallback).
Let'stakethisfurtherandcreatearoutethattakesaJSONrequestbody:
//continued
router.post('/sum',function(req,res){
constvalues=req.body.values;
res.send({
result:values.reduce(function(a,b){
returna+b;
},0)
});
})
.body(joi.object({
values:joi.array().items(joi.number().required()).required()
}).required(),'Valuestoaddtogether.')
.response(joi.object({
result:joi.number().required()
}).required(),'Sumoftheinputvalues.')
.summary('Addupnumbers')
.description('Calculatesthesumofanarrayofnumbervalues.');
Notethatweused posttodefinethisrouteinsteadof get(whichdoesnotsupportrequestbodies).TryingtosendaGETrequestto
thisroute'sURL(intheabsenceofa getrouteforthesamepath)willresultinFoxxrespondingwithanappropriateerrorresponse,
indicatingthesupportedHTTPmethods.
AsthisroutenotonlyexpectsaJSONobjectasinputbutalsorespondswithaJSONobjectasoutputweneedtodefinetwoschemas.
Wedon'tstrictlyneedaresponseschemabutithelpsdocumentingwhattherouteshouldbeexpectedtorespondwithandwillshowup
intheAPIdocumentation.
Becausewe'repassingaschematothe responsemethodwedon'tneedtoexplicitlytellFoxxwearesendingaJSONresponse.The
presenceofaschemaintheabsenceofacontenttypealwaysimplieswewantJSON.Thoughwecouldjustadd ["application/json"]
asanadditionalargumentaftertheschemaifwewantedtomakethismoreexplicit.
The bodymethodworksthesamewayasthe responsemethodexcepttheschemawillbeusedtovalidatetherequestbody.Ifthe
requestbodycan'tbeparsedasJSONordoesn'tmatchtheschema,Foxxwillrejecttherequestwithanappropriateerrorresponse.
Creatingcollections
TherealpowerofFoxxcomesfrominteractingwiththedatabaseitself.Inordertobeabletouseacollectionfromwithinourservice,we
shouldfirstmakesurethatthecollectionactuallyexists.Therightplacetocreatecollectionsyourserviceisgoingtouseisinasetup
script,whichFoxxwillexecuteforyouwheninstallingorupdatingtheservice.
Firstcreateanewfoldercalled"scripts"intheservicefolder,whichwillbewhereourscriptsaregoingtolive.Forsimplicity'ssake,our
setupscriptwillliveinafilecalled setup.jsinsidethatfolder:
//continued
'usestrict';
constdb=require('@arangodb').db;
constcollectionName='myFoxxCollection';
if(!db._collection(collectionName)){
db._createDocumentCollection(collectionName);
}
Thescriptusesthe dbobjectfromthe @arangodbmodule,whichletsusinteractwiththedatabasetheFoxxservicewasinstalledin
andthecollectionsinsidethatdatabase.Becausethescriptmaybeexecutedmultipletimes(i.e.wheneverweupdatetheserviceorwhen
theserverisrestarted)weneedtomakesurewedon'taccidentallytrytocreatethesamecollectiontwice(whichwouldresultinan
exception);wedothatbyfirstcheckingwhetheritalreadyexistsbeforecreatingit.
179
Gettingstarted
The _collectionmethodlooksupacollectionbynameandreturns nullifnocollectionwiththatnamewasfound.The
_createDocumentCollectionmethodcreatesanewdocumentcollectionbyname( _createEdgeCollectionalsoexistsandworks
analogouslyforedgecollections).
Note:Becausewehavehardcodedthecollectionname,multiplecopiesoftheserviceinstalledalongsideeachotherinthesamedatabase
willsharethesamecollection.Becausethismaynotalwaysbewhatyouwant,theFoxxcontextalsoprovidesthe collectionName
methodwhichappliesamountpointspecificprefixtoanygivencollectionnametomakeituniquetotheservice.Italsoprovidesthe
collectionmethod,whichbehavesalmostexactlylike db._collectionexceptitalsoappliestheprefixbeforelookingthecollection
up.
Nextweneedtotellourserviceaboutthescriptbyaddingittothemanifestfile:
{
"engines":{
"arangodb":"^3.0.0"
},
"main":"index.js",
"scripts":{
"setup":"scripts/setup.js"
}
}
Theonlythingthathaschangedisthatweaddeda"scripts"fieldspecifyingthepathofthesetupscriptwejustwrote.
Gobacktothewebinterfaceandupdatetheservicewithournewcode,thenchecktheCollectionstab.Ifeverythingworkedright,you
shouldseeanewcollectioncalled"myFoxxCollection".
Accessingcollections
Let'sexpandourservicebyaddingafewmoreroutestoour index.js:
//continued
constdb=require('@arangodb').db;
consterrors=require('@arangodb').errors;
constfoxxColl=db._collection('myFoxxCollection');
constDOC_NOT_FOUND=errors.ERROR_ARANGO_DOCUMENT_NOT_FOUND.code;
router.post('/entries',function(req,res){
constdata=req.body;
constmeta=foxxColl.save(req.body);
res.send(Object.assign(data,meta));
})
.body(joi.object().required(),'Entrytostoreinthecollection.')
.response(joi.object().required(),'Entrystoredinthecollection.')
.summary('Storeanentry')
.description('Storesanentryinthe"myFoxxCollection"collection.');
router.get('/entries/:key',function(req,res){
try{
constdata=foxxColl.document(req.pathParams.key);
res.send(data)
}catch(e){
if(!e.isArangoError||e.errorNum!==DOC_NOT_FOUND){
throwe;
}
res.throw(404,'Theentrydoesnotexist',e);
}
})
.pathParam('key',joi.string().required(),'Keyoftheentry.')
.response(joi.object().required(),'Entrystoredinthecollection.')
.summary('Retrieveanentry')
.description('Retrievesanentryfromthe"myFoxxCollection"collectionbykey.');
We'reusingthe saveand documentmethodsofthecollectionobjecttostoreandretrievedocumentsinthecollectionwecreatedinour
setupscript.Becausewedon'tcarewhatthedocumentslooklikeweallowanyattributesontherequestbodyandjustacceptanobject.
180
Gettingstarted
BecausethekeywillbeautomaticallygeneratedbyArangoDBwhenonewasn'tspecifiedintherequestbody,we'reusing
Object.assigntoapplytheattributesofthemetadataobjectreturnedbythe savemethodtothedocumentbeforereturningitfrom
ourfirstroute.
The documentmethodreturnsadocumentinacollectionbyits _keyor _id.Howeverwhennomatchingdocumentexistsitthrows
an ArangoErrorexception.BecausewewanttoprovideamoredescriptiveerrormessagethanArangoDBdoesoutofthebox,weneed
tohandlethaterrorexplicitly.
All ArangoErrorexceptionshaveatruthyattribute isArangoErrorthathelpsyourecognizingtheseerrorswithouthavingtoworry
about instanceofchecks.Theyalsoprovidean errorNumandan errorMessage.Ifyouwanttocheckforspecificerrorsyoucanjust
importthe errorsobjectfromthe @arangodbmoduleinsteadofhavingtomemorizenumericerrorcodes.
Insteadofdefiningourownresponselogicfortheerrorcasewejustuse res.throw,whichmakestheresponseobjectthrowan
exceptionFoxxcanrecognizeandconverttotheappropriateserverresponse.WealsopassalongtheexceptionitselfsoFoxxcanprovide
morediagnosticinformationwhenwewantitto.
Wecouldextendthepostroutetosupportarraysofobjectsaswell,eachobjectfollowingacertainschema:
//storeschemainvariabletomakeitre-usable,see.body()
constdocSchema=joi.object().required().keys({
name:joi.string().required(),
age:joi.number().required()
}).unknown();//allowadditionalattributes
router.post('/entries',function(req,res){
constmultiple=Array.isArray(req.body);
constbody=multiple?req.body:[req.body];
letdata=[];
for(vardocofbody){
constmeta=foxxColl.save(doc);
data.push(Object.assign(doc,meta));
}
res.send(multiple?data:data[0]);
})
.body(joi.alternatives().try(
docSchema,
joi.array().items(docSchema)
),'Entryorentriestostoreinthecollection.')
.response(joi.alternatives().try(
joi.object().required(),
joi.array().items(joi.object().required())
),'Entryorentriesstoredinthecollection.')
.summary('Storeentryorentries')
.description('Storeasingleentryormultipleentriesinthe"myFoxxCollection"collection.');
Writingdatabasequeries
Storingandretrievingentriesisfine,butrightnowwehavetomemorizeeachkeywhenwecreateanentry.Let'saddaroutethatgivesus
alistofthekeysofallentriessowecanusethosetolookanentryupindetail.
Thenaïveapproachwouldbetousethe toArray()methodtoconverttheentirecollectiontoanarrayandjustreturnthat.Butwe're
onlyinterestedinthekeysandtheremightpotentiallybesomanyentriesthatfirstretrievingeverysingledocumentmightgetunwieldy.
Let'swriteashortAQLquerytodothisinstead:
//continued
constaql=require('@arangodb').aql;
router.get('/entries',function(req,res){
constkeys=db._query(aql`
FORentryIN${foxxColl}
RETURNentry._key
`);
res.send(keys);
})
.response(joi.array().items(
joi.string().required()
181
Gettingstarted
).required(),'Listofentrykeys.')
.summary('Listentrykeys')
.description('Assemblesalistofkeysofentriesinthecollection.');
Herewe'reusingtwonewthings:
The _querymethodexecutesanAQLqueryintheactivedatabase.
The aqltemplatestringhandlerallowsustowritemulti-lineAQLqueriesandalsohandlesqueryparametersandcollectionnames.
Insteadofhardcodingthenameofthecollectionwewanttouseinthequerywecansimplyreferencethe foxxCollvariablewedefined
earlier--itrecognizesthevalueasanArangoDBcollectionobjectandknowswearespecifyingacollectionratherthanaregularvalue
eventhoughAQLdistinguishesbetweenthetwo.
Note:Ifyouaren'tusedtoJavaScripttemplatestringsandtemplatestringhandlersjustthinkof aqlasafunctionthatreceivesthe
multilinestringsplitatevery ${}expressionaswellasanarrayofthevaluesofthoseexpressions--that'sactuallyallthereistoit.
Alternatively,here'saversionwithouttemplatestrings(noticehowmuchcleanerthe aqlversionwillbeincomparisonwhenyouhave
multiplevariables):
constkeys=db._query(
'FORentryIN@@collRETURNentry._key',
{'@coll':foxxColl}
);
Nextsteps
YounowknowhowtocreateaFoxxservicefromscratch,howtohandleuserinputandhowtoaccessthedatabasefromwithinyour
Foxxservicetostore,retrieveandquerydatayoustoreinsideArangoDB.ThisshouldallowyoutobuildmeaningfulAPIsforyourown
applicationsbuttherearemanymorethingsyoucandowithFoxx:
Needtogofaster?Turnondevelopmentmodeandhackonyourcoderightontheserver.
Concernedaboutsecurity?Youcouldaddauthenticationtoyourservicetoprotectaccesstothedatabeforeitevenleavesthe
database.
Writingasinglepageapp?YoucouldstoresomebasicassetsrightinsideyourFoxxservice.
Needtointegrateexternalservices?YoucanmakeHTTPrequestsfrominsideFoxxandusequeuedjobstoperformthatworkinthe
background.
Tiredofreinventingthewheel?Learnaboutdependencies.
Everythingbroken?Youcanwriteteststomakesureyourlogicremainssound.
182
Servicemanifest
Manifestfiles
Everyservicecomeswitha manifest.jsonfileprovidingmetadata.Thefollowingfieldsareallowedinmanifests:
configuration: Object(optional)
Anobjectdefiningtheconfigurationoptionsthisservicerequires.
defaultDocument: string(optional)
Ifspecified,the /(root)routeoftheservicewillautomaticallyredirecttothegivenrelativepath,e.g.:
"defaultDocument":"index.html"
ThiswouldhavethesameeffectascreatingthefollowingrouteinJavaScript:
constcreateRouter=require('@arangodb/foxx/router');
constindexRouter=createRouter();
indexRouter.all('/',function(req,res){
res.redirect('index.html');
});
module.context.use(indexRouter);
Note:Asof3.0.0thisfieldcansafelybeomitted;thevaluenolongerdefaultsto "index.html".
dependencies: Object(optional)andprovides: Object(optional)
Objectsspecifyingotherservicesthisservicehasasdependenciesandwhatdependenciesitcanprovidetootherservices.
engines: Object(optional)
AnobjectindicatingthesemanticversionrangesofArangoDB(orcompatibleenvironments)theservicewillbecompatiblewith,
e.g.:
"engines":{
"arangodb":"^3.0.0"
}
ThisshouldcorrectlyindicatetheminimumversionofArangoDBtheservicehasbeentestedagainst.Foxxmaintainsastrict
semanticversioningpolicyasofArangoDB3.0.0soitisgenerallysafetousesemverranges(e.g. ^3.0.0tomatchanyversion
greaterorequalto 3.0.0andbelow 4.0.0)formaximumcompatibility.
files: Object(optional)
Anobjectdefiningfileassetsservedbythisservice.
lib: string(Default: ".")
TherelativepathtotheFoxxJavaScriptfilesintheservice,e.g.:
"lib":"lib"
Thiswouldresultinthemainentrypoint(seebelow)andotherJavaScriptpathsbeingresolvedasrelativetothe libfolderinside
theservicefolder.
main: string(optional)
Therelativepathtothemainentrypointofthisservice(relativetolib,seeabove),e.g.:
"main":"index.js"
ThiswouldresultinFoxxloadingandexecutingthefile index.jswhentheserviceismountedorstarted.
183
Servicemanifest
Note:whileitistechnicallypossibletoomitthisfield,youwilllikelywanttoprovideanentrypointtoyourserviceasthisisthe
onlywaytoexposeHTTProutesorexportaJavaScriptAPI.
scripts: Object(optional)
Anobjectdefiningnamedscriptsprovidedbythisservice,whichcaneitherbeuseddirectlyorasqueuedjobsbyotherservices.
tests: stringor Array<string>(optional)
ApathorlistofpathsofJavaScripttestsprovidedforthisservice.
Additionallymanifestscanprovidethefollowingmetadata:
author: string(optional)
Thefullnameoftheauthoroftheservice(i.e.you).Thiswillbeshowninthewebinterface.
contributors: Array<string>(optional)
Alistofnamesofpeoplethathavecontributedtothedevelopmentoftheserviceinsomeway.Thiswillbeshownintheweb
interface.
description: string(optional)
Ahuman-readabledescriptionoftheservice.Thiswillbeshowninthewebinterface.
keywords: Array<string>(optional)
Alistofkeywordsthathelpcategorizethisservice.ThisisusedbytheFoxxStoreinstallerstoorganizeservices.
license: string(optional)
Astringidentifyingthelicenseunderwhichtheserviceispublished,ideallyintheformofanSPDXlicenseidentifier.Thiswillbe
showninthewebinterface.
name: string(optional)
ThenameoftheFoxxservice.AllowedcharactersareA-Z,0-9,theASCIIhyphen( -)andunderscore( _)characters.Thename
mustnotstartwithanumber.Thiswillbeshowninthewebinterface.
thumbnail: string(optional)
Thefilenameofathumbnailthatwillbeusedalongsidetheserviceinthewebinterface.ThisshouldbeaJPEGorPNGimagethat
looksgoodatsizes50x50and160x160.
version: string(optional)
TheversionnumberoftheFoxxservice.Theversionnumbermustfollowthesemanticversioningformat.Thiswillbeshowninthe
webinterface.
Examples
{
"name":"example-foxx-service",
"version":"3.0.0-dev",
"license":"MIT",
"description":"Anexampleservicewitharelativelyfull-featuredmanifest.",
"thumbnail":"foxx-icon.png",
"keywords":["demo","service"],
"author":"ArangoDBGmbH",
"contributors":[
"AlanPlum<alan@arangodb.example>"
],
"lib":"dist",
"main":"entry.js",
"defaultDocument":"welcome.html",
"engines":{
"arangodb":"^3.0.0"
},
"files":{
184
Servicemanifest
"welcome.html":"assets/index.html",
"hello.jpg":"assets/hello.jpg"
"world.jpg":{
"path":"assets/world.jpg",
"type":"image/jpeg",
"gzip":false
}
},
"tests":"dist/**.spec.js"
}
185
Servicecontext
Foxxservicecontext
Theservicecontextprovidesaccesstomethodsandattributesthatarespecifictoagivenservice.InaFoxxservicethecontextisgenerally
availableasthe module.contextvariable.Withinarouter'srequesthandlertherequestandresponseobjects' contextattributealso
provideaccesstothecontextoftheservicetheroutewasmountedin(whichmaybedifferentfromtheonetheroutehandlerwasdefined
in).
Examples
//inservice/my-foxx-1
constcreateRouter=require('@arangodb/foxx/router');
constrouter=createRouter();
//Seethechapterondependenciesformoreinfoon
//howexportsanddependenciesworkacrossservices
module.exports={routes:router};
router.get(function(req,res){
module.context.mount==='/my-foxx-1';
req.context.mount==='/my-foxx-2';
res.write('Hellofrommy-foxx-1');
});
//inservice/my-foxx-2
constcreateRouter=require('@arangodb/foxx/router');
constrouter2=createRouter();
module.context.use(router2);
router2.post(function(req,res){
module.context.mount==='/my-foxx-2';
req.context.mount==='/my-foxx-2';
res.write('Hellofrommy-foxx-2');
});
constrouter1=module.context.dependencies.myFoxx1.routes;
module.context.use(router1);
Theservicecontextspecifiesthefollowingproperties:
argv: any
Anyargumentspassedinifthecurrentfilewasexecutedasascriptorqueuedjob.
basePath: string
Thefilesystempathoftheservice,i.e.thefolderinwhichtheservicewasinstalledtobyArangoDB.
baseUrl: string
ThebaseURLoftheservice,relativetotheArangoDBserver,e.g. /_db/_system/my-foxx.
collectionPrefix: string
TheprefixthatwillbeusedbycollectionandcollectionNametoderivethenamesofservice-specificcollections.Thisisderivedfrom
theservice'smountpoint,e.g. /my-foxxbecomes my_foxx.
configuration: Object
Configurationoptionsfortheservice.
dependencies: Object
Configureddependenciesfortheservice.
isDevelopment: boolean
Indicateswhethertheserviceisrunningindevelopmentmode.
186
Servicecontext
isProduction: boolean
TheinverseofisDevelopment.
manifest: Object
Theparsedmanifestfileoftheservice.
mount: string
Themountpointoftheservice,e.g. /my-foxx.
apiDocumentation
module.context.apiDocumentation([options]):Function
DEPRECATED
CreatesarequesthandlerthatservestheAPIdocumentation.
Note:ThismethodhasbeendeprecatedinArangoDB3.1andreplacedwiththemorestraightforward createDocumentationRouter
methodprovidingthesamefunctionality.
Arguments
See createDocumentationRouterbelow.
Examples
//ServetheAPIdocsforthecurrentservice
router.get('/docs/*',module.context.apiDocumentation());
//Notethatthepathmustendwithawildcard
//andtheroutemustuseHTTPGET.
createDocumentationRouter
module.context.createDocumentationRouter([options]):Router
CreatesarouterthatservestheAPIdocumentation.
Note:Theroutercanbemountedlikeanyotherchildrouter(seeexamplesbelow).
Arguments
options: Object(optional)
Anobjectwithanyofthefollowingproperties:
mount: string(Default: module.context.mount)
Themountpathoftheservicetoservethedocumentationof.
indexFile: string(Default: "index.html")
FilenameoftheHTM LfileservingtheAPIdocumentation.
swaggerRoot: string(optional)
FullpathofthefoldercontainingtheSwaggerassetsandtheindexFile.DefaultstotheSwaggerassetsusedbytheweb
interface.
before: Function(optional)
Afunctionthatwillbeexecutedbeforearequestishandled.
Ifthefunctionreturns falsetherequestwillnotbeprocessedanyfurther.
Ifthefunctionreturnsanobject,itsattributeswillbeusedtooverridetheoptionsforthecurrentrequest.
187
Servicecontext
Anyotherreturnvaluewillbeignored.
Ifoptionsisafunctionitwillbeusedasthebeforeoption.
IfoptionsisastringitwillbeusedastheswaggerRootoption.
ReturnsaFoxxrouter.
Examples
//ServetheAPIdocsforthecurrentservice
router.use('/docs',module.context.createDocumentationRouter());
//--or-//ServetheAPIdocsfortheservicetherouterismountedin
router.use('/docs',module.context.createDocumentationRouter(function(req){
return{mount:req.context.mount};
}));
//--or-//ServetheAPIdocsonlyforusersauthenticatedwithArangoDB
router.use('/docs',module.context.createDocumentationRouter(function(req,res){
if(req.suffix==='swagger.json'&&!req.arangoUser){
res.throw(401,'Notauthenticated');
}
}));
collection
module.context.collection(name):ArangoCollection|null
PassesthegivennametocollectionName,thenlooksupthecollectionwiththeprefixedname.
Arguments
name: string
Unprefixednameoftheservice-specificcollection.
Returnsacollectionor nullifnocollectionwiththeprefixednameexists.
collectionName
module.context.collectionName(name):string
PrefixesthegivennamewiththecollectionPrefixforthisservice.
Arguments
name: string
Unprefixednameoftheservice-specificcollection.
Returnstheprefixedname.
Examples
module.context.mount==='/my-foxx'
module.context.collectionName('doodads')==='my_foxx_doodads'
file
module.context.file(name,[encoding]):Buffer|string
PassesthegivennametofileName,thenloadsthefilewiththeresultingname.
188
Servicecontext
Arguments
name: string
Nameofthefiletoload,relativetothecurrentservice.
encoding: string(optional)
Encodingofthefile,e.g. utf-8.Ifomittedthefilewillbeloadedasarawbufferinsteadofastring.
Returnsthefile'scontents.
fileName
module.context.fileName(name):string
Resolvesthegivenfilenamerelativetothecurrentservice.
Arguments
name: string
Nameofthefile,relativetothecurrentservice.
Returnstheabsolutefilepath.
use
module.context.use([path],router):Endpoint
M ountsagivenrouterontheservicetoexposetherouter'sroutesontheservice'smountpoint.
Arguments
path: string(Default: "/")
Pathtomounttherouterat,relativetotheservice'smountpoint.
router: Router|Middleware
Arouterormiddlewaretomount.
ReturnsanEndpointforthegivenrouterormiddleware.
Note:M ountingservicesatruntime(e.g.withinrequesthandlersorqueuedjobs)isnotsupported.
189
Configuration
Foxxconfiguration
Foxxservicescandefineconfigurationparameterstomakethemmorere-usable.
The configurationobjectmapsnamestoconfigurationparameters:
Thekeyisthenameunderwhichtheparameterwillbeavailableontheservicecontext's configurationproperty.
Thevalueisaparameterdefinition.
Theparameterdefinitioncanhavethefollowingproperties:
description: string
Humanreadabledescriptionoftheparameter.
type: string(Default: "string")
Typeoftheconfigurationparameter.Supportedvaluesare:
"integer"or "int":anyfiniteintegernumber.
"boolean"or "bool":thevalues trueor false.
"number":anyfinitedecimalorintegernumber.
"string":anystringvalue.
"json":anywell-formedJSONvalue.
"password":likestringbutwillbedisplayedasamaskedinputfieldinthewebfrontend.
default: any
Defaultvalueoftheconfigurationparameter.
required:(Default: true)
Whethertheparameterisrequired.
Iftheconfigurationhasparametersthatdonotspecifyadefaultvalue,youneedtoconfiguretheservicebeforeitbecomesactive.Inthe
meantimeafallbackservicelicationwillbemountedthatrespondstoallrequestswithaHTTP500statuscodeindicatingaserver-side
error.
TheconfigurationparametersofamountedservicecanbeadjustedfromthewebinterfacebyclickingtheConfigurationbuttoninthe
servicedetails.
Examples
"configuration":{
"currency":{
"description":"Currencysymboltouseforpricesintheshop.",
"default":"$",
"type":"string"
},
"secretKey":{
"description":"Secretkeytouseforsigningsessiontokens.",
"type":"password"
}
}
190
Dependencies
Dependencymanagement
Therearetwothingscommonlycalled"dependencies"inFoxx:
M oduledependencies,i.e.dependenciesonexternalJavaScriptmodules(e.g.fromthepublicnpmregistry)
Foxxdependencies,i.e.dependenciesbetweenFoxxservices
Let'slookattheminmoredetail:
Moduledependencies
Youcanusethe node_modulesfoldertobundlethird-partyFoxx-compatiblenpmandNode.jsmoduleswithyourFoxxservice.
Typicallythisisachievedbyaddinga package.jsonfiletoyourprojectspecifyingnpmdependenciesusingthe dependenciesattribute
andinstallingthemwiththenpmcommand-linetool.
M akesuretoincludetheactual node_modulesfolderinyourFoxxservicebundleasArangoDBwillnotdoanythingspecialtoinstall
thesedependencies.Alsokeepinmindthatbundlingextraneousmoduleslikedevelopmentdependenciesmaybloatthefilesizeofyour
Foxxservicebundle.
Compatibilitycaveats
UnlikeJavaScriptinbrowsersorNode.js,theJavaScriptenvironmentinArangoDBissynchronous.Thismeansanymodulesthat
dependonasynchronousbehaviourlikepromisesor setTimeoutwillnotbehavecorrectlyinArangoDBorFoxx.Additionallyunlike
Node.jsArangoDBdoesnotsupportnativeextensions.AllmoduleshavetobeimplementedinpureJavaScript.
WhileArangoDBprovidesalotofcompatibilitycodetosupportmoduleswrittenforNode.js,someNode.jsbuilt-inmodulescannotbe
providedbyArangoDB.ForacloserlookattheNode.jsmodulesArangoDBdoesordoesnotprovidecheckouttheappendixon
JavaScriptmodules.
Alsonotethattheserestrictionsnotonlyapplyonthemodulesyouwishtoinstallbutalsothedependenciesofthosemodules.Asarule
ofthumb:moduleswrittentoworkinNode.jsandthebrowserthatdonotrelyonasyncbehaviourshouldgenerallywork;modulesthat
relyonnetworkorfilesystemI/Oormakeheavyuseofasyncbehaviourmostlikelywillnot.
Foxxdependencies
Foxxdependenciescanbedeclaredinaservice'smanifestusingthe providesand dependenciesfields:
providesliststhedependenciesagivenserviceprovides,i.e.whichAPIsitclaimstobecompatiblewith
dependenciesliststhedependenciesagivenserviceuses,i.e.whichAPIsitsdependenciesneedtobecompatiblewith
Adependencynameshouldgenerallyusethesameformatasanamespaced(org-scoped)NPM module,e.g. @foxx/sessions.
DependencynamesrefertotheexternalJavaScriptAPIofaserviceratherthanspecificservicesimplementingthoseAPIs.Some
dependencynamesdefinedbyofficiallymaintainedservicesare:
@foxx/auth(version 1.0.0)
@foxx/api-keys(version 1.0.0)
@foxx/bugsnag(versions 1.0.0and 2.0.0)
@foxx/mailgun(versions 1.0.0and 2.0.0)
@foxx/postageapp(versions 1.0.0and 2.0.0)
@foxx/postmark(versions 1.0.0and 2.0.0)
@foxx/sendgrid(versions 1.0.0and 2.0.0)
@foxx/oauth2(versions 1.0.0and 2.0.0)
@foxx/segment-io(versions 1.0.0and 2.0.0)
@foxx/sessions(versions 1.0.0and 2.0.0)
@foxx/users(versions 1.0.0, 2.0.0and 3.0.0)
191
Dependencies
A providesdefinitionmapseachprovideddependency'snametotheprovidedversion:
"provides":{
"@foxx/auth":"1.0.0"
}
A dependenciesdefinitionmapsthelocalaliasofagivendependencyagainstitsnameandthesupportedversionrange(eitherasa
JSONobjectorashorthandstring):
"dependencies":{
"mySessions":"@foxx/sessions:^2.0.0",
"myAuth":{
"name":"@foxx/auth",
"version":"^1.0.0",
"description":"Thisdescriptionisentirelyoptional.",
"required":false
}
}
Dependenciescanbeconfiguredfromthewebinterfaceinaservice'ssettingstabusingtheDependenciesbutton.
Thevalueforeachdependencyshouldbethedatabase-relativemountpathoftheservice(includingtheleadingslash).Inordertobe
usableasthedependencyofanotherservicebothservicesneedtobemountedinthesamedatabase.Aservicecanbeusedtoprovide
multipledependenciesforthesameservice(aslongastheexpectedJavaScriptAPIsdon'tconflict).
Aservicethathasunconfiguredrequireddependenciescannotbeuseduntilallofitsdependencieshavebeenconfigured.
Itispossibletospecifythemountpathofaservicethatdoesnotactuallydeclarethedependencyasprovided.Thereiscurrentlyno
validationbeyondthemanifestformats.
Whenaserviceusesanothermountedserviceasadependencythedependency's mainentryfile's exportsobjectbecomesavailablein
the module.context.dependenciesobjectoftheotherservice:
Examples
ServiceAandServiceBaremountedinthesamedatabase.ServiceBhasadependencywiththelocalalias "greeter".Thedependency
isconfiguredtousethemountpathofServiceA.
//EntryfileofServiceA
module.exports={
sayHi(){
return'Hello';
}
};
//SomewhereinServiceB
constgreeter=module.context.dependencies.greeter;
res.write(greeter.sayHi());
192
Routers
Routers
constcreateRouter=require('@arangodb/foxx/router');
RoutersletyoudefineroutesthatextendArangoDB'sHTTPAPIwithcustomendpoints.
Routersneedtobemountedusingthe usemethodofaservicecontexttoexposetheirHTTProutesataservice'smountpath.
Youcanpassroutersbetweenservicesmountedinthesamedatabaseasdependencies.Youcanevennestrouterswithineachother.
Creatingarouter
createRouter():Router
Thisreturnsanew,cleanrouterobjectthathasnotyetbeenmountedintheserviceandcanbeexportedlikeanyotherobject.
Requesthandlers
router.get([path],[...middleware],handler,[name]):Endpoint
router.post([path],[...middleware],handler,[name]):Endpoint
router.put([path],[...middleware],handler,[name]):Endpoint
router.patch([path],[...middleware],handler,[name]):Endpoint
router.delete([path],[...middleware],handler,[name]):Endpoint
router.all([path],[...middleware],handler,[name]):Endpoint
Thesemethodsletyouspecifyroutesontherouter.The allmethoddefinesaroutethatwillmatchanysupportedHTTPverb,the
othermethodsdefineroutesthatonlymatchtheHTTPverbwiththesamename.
Arguments
path: string(Default: "/")
ThepathoftherequesthandlerrelativetothebasepaththeRouterismountedat.Ifomitted,therequesthandlerwillhandle
requeststothebasepathoftheRouter.Forinformationondefiningdynamicroutesseethesectiononpathparametersinthe
chapteronrouterendpoints.
middleware: Function(optional)
Zeroormoremiddlewarefunctionsthattakethefollowingarguments:
req: Request
Anincomingserverrequestobject.
res: Response
Anoutgoingserverresponseobject.
next: Function
Acallbackthatpassescontrolovertothenextmiddlewarefunctionandreturnswhenthatfunctionhascompleted.
Ifatruthyargumentispassed,thatargumentwillbethrownasanerror.
Ifthereisnonextmiddlewarefunction,the handlerwillbeinvokedinstead(seebelow).
handler: Function
Afunctionthattakesthefollowingarguments:
req: Request
Anincomingserverrequestobject.
193
Routers
res: Response
Anoutgoingserverresponse.
name: string(optional)
AnamethatcanbeusedtogenerateURLsfortheendpoint.Formoreinformationseethe reversemethodoftherequestobject.
ReturnsanEndpointfortheroute.
Examples
Simpleindexroute:
router.get(function(req,res){
res.set('content-type','text/plain');
res.write('HelloWorld!');
});
RestrictingaccesstoauthenticatedArangoDBusers:
router.get('/secrets',function(req,res,next){
if(req.arangoUser){
next();
}else{
res.throw(404,'Secrets?Whatsecrets?');
}
},function(req,res){
res.download('allOurSecrets.zip');
});
M ultiplemiddlewarefunctions:
functioncounting(req,res,next){
if(!req.counter)req.counter=0;
req.counter++;
next();
req.counter--;
}
router.get(counting,counting,counting,function(req,res){
res.json({counter:req.counter});//{"counter":3}
});
Mountingchildroutersandmiddleware
router.use([path],middleware,[name]):Endpoint
The usemethodletsyoumountachildrouterormiddlewareatagivenpath.
Arguments
path: string(optional)
ThepathofthemiddlewarerelativetothebasepaththeRouterismountedat.Ifomitted,themiddlewarewillhandlerequeststothe
basepathoftheRouter.Forinformationondefiningdynamicroutesseethesectiononpathparametersinthechapteronrouter
endpoints.
middleware: Router|Middleware
Anunmountedrouterobjectoramiddleware.
name: string(optional)
AnamethatcanbeusedtogenerateURLsforendpointsofthisrouter.Formoreinformationseethe reversemethodofthe
requestobject.HasnoeffectifhandlerisaM iddleware.
ReturnsanEndpointforthemiddlewareorchildrouter.
194
Routers
195
Endpoints
Endpoints
Endpointsarereturnedbythe use, allandHTTPverb(e.g. get, post)methodsofroutersaswellasthe usemethodofthe
servicecontext.Theycanbeusedtoattachmetadatatomountedroutes,middlewareandchildroutersthataffectshowrequestsand
responsesareprocessedorprovidesAPIdocumentation.
Endpointsshouldonlybeusedtoinvokethefollowingmethods.Endpointmethodscanbechainedtogether(eachmethodreturnsthe
endpointitself).
header
endpoint.header(name,[schema],[description]):this
Definesarequestheaderrecognizedbytheendpoint.Anyadditionalnon-definedheaderswillbetreatedasoptionalstringvalues.The
definitionswillalsobeshownintheroutedetailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisheaderdefinitionunlessoverridden.
Arguments
name: string
Nameoftheheader.Thisshouldbeconsideredcaseinsensitiveasallheadernameswillbeconvertedtolowercase.
schema: Schema(optional)
Aschemadescribingtheformatoftheheadervalue.Thiscanbeajoischemaoranythingthathasacompatible validatemethod.
Thevalueofthisheaderwillbesettothe valuepropertyofthevalidationresult.Avalidationfailurewillresultinanautomatic
400(BadRequest)errorresponse.
description: string(optional)
AhumanreadablestringthatwillbeshownintheAPIdocumentation.
Returnstheendpoint.
Examples
router.get(/*...*/)
.header('arangoVersion',joi.number().min(30000).default(30000));
pathParam
endpoint.pathParam(name,[schema],[description]):this
Definesapathparameterrecognizedbytheendpoint.Pathparametersareexpectedtobefilledaspartoftheendpoint'smountpath.
Anyadditionalnon-definedpathparameterswillbetreatedasoptionalstringvalues.Thedefinitionswillalsobeshownintheroute
detailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisparameterdefinitionunlessoverridden.
Arguments
name: string
Nameoftheparameter.
schema: Schema(optional)
Aschemadescribingtheformatoftheparameter.Thiscanbeajoischemaoranythingthathasacompatible validatemethod.
196
Endpoints
Thevalueofthisparameterwillbesettothe valuepropertyofthevalidationresult.Avalidationfailurewillresultintheroute
failingtomatchandbeingignored(resultingina404(NotFound)errorresponseifnootherroutesmatch).
description: string(optional)
AhumanreadablestringthatwillbeshownintheAPIdocumentation.
Returnstheendpoint.
Examples
router.get('/some/:num/here',/*...*/)
.pathParam('num',joi.number().required());
queryParam
endpoint.queryParam(name,[schema],[description]):this
Definesaqueryparameterrecognizedbytheendpoint.Anyadditionalnon-definedqueryparameterswillbetreatedasoptionalstring
values.ThedefinitionswillalsobeshownintheroutedetailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisparameterdefinitionunlessoverridden.
Arguments
name: string
Nameoftheparameter.
schema: Schema(optional)
Aschemadescribingtheformatoftheparameter.Thiscanbeajoischemaoranythingthathasacompatible validatemethod.
Thevalueofthisparameterwillbesettothe valuepropertyofthevalidationresult.Avalidationfailurewillresultinan
automatic400(BadRequest)errorresponse.
description: string(optional)
AhumanreadablestringthatwillbeshownintheAPIdocumentation.
Returnstheendpoint.
Examples
router.get(/*...*/)
.queryParam('num',joi.number().required());
body
endpoint.body([model],[mimes],[description]):this
Definestherequestbodyrecognizedbytheendpoint.Therecanonlybeonerequestbodydefinitionperendpoint.Thedefinitionwill
alsobeshownintheroutedetailsintheAPIdocumentation.
Intheabsenceofarequestbodydefinition,therequestobject'sbodypropertywillbeinitializedtotheunprocessedrawBodybuffer.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisbodydefinitionunlessoverridden.Iftheendpointisamiddleware,
therequestbodywillonlybeparsedonce(i.e.theM IM Etypesoftheroutematchingthesamerequestwillbeignoredbutthebodywill
stillbevalidatedagain).
Arguments
model: Model|Schema|null(optional)
Amodelorjoischemadescribingtherequestbody.Avalidationfailurewillresultinanautomatic400(BadRequest)errorresponse.
197
Endpoints
Ifthevalueisamodelwitha fromClientmethod,thatmethodwillbeappliedtotheparsedrequestbody.
Ifthevalueisaschemaoramodelwithaschema,theschemawillbeusedtovalidatetherequestbodyandthe valuepropertyof
thevalidationresultoftheparsedrequestbodywillbeusedinsteadoftheparsedrequestbodyitself.
IfthevalueisamodeloraschemaandtheM IM Etypehasbeenomitted,theM IM EtypewilldefaulttoJSONinstead.
Ifthevalueisexplicitlysetto null,norequestbodywillbeexpected.
Ifthevalueisanarraycontainingexactlyonemodelorschema,therequestbodywillbetreatedasanarrayofitemsmatchingthat
modelorschema.
mimes: Array<string>(optional)
AnarrayofM IM Etypestheroutesupports.
Commonnon-mimealiaseslike"json"or"html"arealsosupportedandwillbeexpandedtotheappropriateM IM Etype(e.g.
"application/json"and"text/html").
IftheM IM EtypeisrecognizedbyFoxxtherequestbodywillbeparsedintotheappropriatestructurebeforebeingvalidated.
CurrentlyonlyJSON, application/x-www-form-urlencodedandmultipartformatsaresupportedinthisway.
IftheM IM EtypeindicatedintherequestheadersdoesnotmatchanyofthesupportedM IM Etypes,thefirstM IM Etypeinthe
listwillbeusedinstead.
Failuretoparsetherequestbodywillresultinanautomatic400(BadRequest)errorresponse.
description: string(optional)
AhumanreadablestringthatwillbeshownintheAPIdocumentation.
Returnstheendpoint.
Examples
router.post('/expects/some/json',/*...*/)
.body(
joi.object().required(),
'ThisimpliesJSON.'
);
router.post('/expects/nothing',/*...*/)
.body(null);//Nobodyallowed
router.post('/expects/some/plaintext',/*...*/)
.body(['text/plain'],'Thisbodywillbeastring.');
response
endpoint.response([status],[model],[mimes],[description]):this
Definesaresponsebodyfortheendpoint.Whenusingtheresponseobject's sendmethodintherequesthandlerofthisroute,the
definitionwiththematchingstatuscodewillbeusedtogeneratetheresponsebody.Thedefinitionswillalsobeshownintheroute
detailsintheAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisresponsedefinitionunlessoverridden.Iftheendpointisa
middleware,thismethodhasnoeffect.
Arguments
status: number|string(Default: 200or 204)
HTTPstatuscodetheresponseappliesto.Ifastringisprovidedinsteadofanumericstatuscodeitwillbeusedtolookupa
numericstatuscodeusingthestatusesmodule.
model: Model|Schema|null(optional)
Amodelorjoischemadescribingtheresponsebody.
198
Endpoints
Ifthevalueisamodelwitha forClientmethod,thatmethodwillbeappliedtothedatapassedto response.sendwithinthe
routeiftheresponsestatuscodematches(butalsoifnostatuscodehasbeenset).
Ifthevalueisaschemaoramodelwithaschema,theactualschemawillnotbeusedtovalidatetheresponsebodyandonlyserves
todocumenttheresponseinmoredetailintheAPIdocumentation.
IfthevalueisamodeloraschemaandtheM IM Etypehasbeenomitted,theM IM EtypewilldefaulttoJSONinstead.
Ifthevalueisexplicitlysetto nullandthestatuscodehasbeenomitted,thestatuscodewilldefaultto 204("nocontent")
insteadof 200.
Ifthevalueisanarraycontainingexactlyonemodelorschema,theresponsebodywillbeanarrayofitemsmatchingthatmodelor
schema.
mimes: Array<string>(optional)
AnarrayofM IM Etypestheroutemightrespondwithforthisstatuscode.
Commonnon-mimealiaseslike"json"or"html"arealsosupportedandwillbeexpandedtotheappropriateM IM Etype(e.g.
"application/json"and"text/html").
Whenusingthe response.sendmethodtheresponsebodywillbeconvertedtotheappropriateM IM Etypeifpossible.
description: string(optional)
Ahuman-readablestringthatbrieflydescribestheresponseandwillbeshownintheendpoint'sdetaileddocumentation.
Returnstheendpoint.
Examples
//Thisexampleonlyprovidesdocumentation
//andimpliesagenericJSONresponsebody.
router.get(/*...*/)
.response(
joi.array().items(joi.string()),
'Alistofdoodadidentifiers.'
);
//Noresponsebodywillbeexpectedhere.
router.delete(/*...*/)
.response(null,'Thedoodadnolongerexists.');
//Anendpointcandefinemultipleresponsetypes
//fordifferentstatuscodes--butnevermorethan
//oneforeachstatuscode.
router.post(/*...*/)
.response('found','Thedoodadislocatedelsewhere.')
.response(201,['text/plain'],'Thedoodadwascreatedsohereisahaiku.');
//Heretheresponsebodywillbesetto
//thequerystring-encodedresultof
//FormModel.forClient({some:'data'})
//becausethestatuscodedefaultsto200.
router.patch(function(req,res){
//...
res.send({some:'data'});
})
.response(FormModel,['application/x-www-form-urlencoded'],'OMG.');
//Inthiscasetheresponsebodywillbesetto
//SomeModel.forClient({some:'data'})because
//thestatuscodehasbeensetto201before.
router.put(function(req,res){
//...
res.status(201);
res.send({some:'data'});
})
.response(201,SomeModel,'Somethingamazinghappened.');
error
199
Endpoints
endpoint.error(status,[description]):this
Documentsanerrorstatusfortheendpoint.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethiserrordescriptionunlessoverridden.Iftheendpointisamiddleware,
thismethodhasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
status: number|string
HTTPstatuscodefortheerror(e.g. 400for"badrequest").Ifastringisprovidedinsteadofanumericstatuscodeitwillbeused
tolookupanumericstatuscodeusingthestatusesmodule.
description: string(optional)
Ahuman-readablestringthatbrieflydescribestheerrorconditionandwillbeshownintheendpoint'sdetaileddocumentation.
Returnstheendpoint.
Examples
router.get(function(req,res){
//...
res.throw(403,'Validationerroratx.y.z');
})
.error(403,'Indicatesthatavalidationhasfailed.');
summary
endpoint.summary(summary):this
Addsashortdescriptiontotheendpoint'sAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethissummaryunlessoverridden.Iftheendpointisamiddleware,this
methodhasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
summary: string
Ahuman-readablestringthatbrieflydescribestheendpointandwillappearnexttotheendpoint'spathinthedocumentation.
Returnstheendpoint.
Examples
router.get(/*...*/)
.summary('Listalldiscombobulateddoodads')
description
endpoint.description(description):this
Addsalongdescriptiontotheendpoint'sAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillusethisdescriptionunlessoverridden.Iftheendpointisamiddleware,this
methodhasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
200
Endpoints
description: string
Ahuman-readablestringthatdescribestheendpointindetailandwillbeshownintheendpoint'sdetaileddocumentation.
Returnstheendpoint.
Examples
//The"dedent"libraryhelpsformatting
//multi-linestringsbyadjustingindentation
//andremovingleadingandtrailingblanklines
constdd=require('dedent');
router.post(/*...*/)
.description(dd`
Thisroutediscombobulatesthedoodadsby
frobnicatingthemoxieoftherequestbody.
`)
deprecated
endpoint.deprecated([deprecated]):this
M arkstheendpointasdeprecated.
Iftheendpointisachildrouter,allroutesofthatrouterwillalsobemarkedasdeprecated.Iftheendpointisamiddleware,thismethod
hasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
deprecated: boolean(Default: true)
Whethertheendpointshouldbemarkedasdeprecated.Ifsetto falsetheendpointwillbeexplicitlymarkedasnotdeprecated.
Returnstheendpoint.
Examples
router.get(/*...*/)
.deprecated();
tag
endpoint.tag(...tags):this
M arkstheendpointwiththegiventagsthatwillbeusedtogrouprelatedroutesinthegeneratedAPIdocumentation.
Iftheendpointisachildrouter,allroutesofthatrouterwillalsobemarkedwiththetags.Iftheendpointisamiddleware,thismethod
hasnoeffect.
ThismethodonlyaffectsthegeneratedAPIdocumentationandhasnotothereffectwithintheserviceitself.
Arguments
tags: string
Oneormorestringsthatwillbeusedtogrouptheendpoint'sroutes.
Returnstheendpoint.
Examples
router.get(/*...*/)
.tag('auth','restricted');
201
Endpoints
202
M iddleware
Middleware
M iddlewareinFoxxreferstofunctionsthatcanbemountedlikeroutesandcanmanipulatetherequestandresponseobjectsbeforeand
aftertherouteitselfisinvoked.Theycanalsobeusedtocontrolaccessortoprovidecommonlogiclikeloggingetc.Unlikeroutes,
middlewareismountedwiththe usemethodlikearouter.
Insteadofafunctionthe usemethodcanalsoacceptanobjectwitha registerfunctionthatwillbepassedtheendpointthe
middlewarewillbemountedatandreturnstheactualmiddlewarefunction.Thisallowsmanipulatingtheendpointbeforecreatingthe
middleware(e.g.todocumentheaders,requestbodies,pathparametersorqueryparameters).
Examples
RestrictaccesstoArangoDB-authenticatedusers:
module.context.use(function(req,res,next){
if(!req.arangoUser){
res.throw(401,'NotauthenticatedwithArangoDB');
}
next();
});
Anytruthyargumentpassedtothe nextfunctionwillbethrownasanerror:
module.context.use(function(req,res,next){
leterr=null;
if(!req.arangoUser){
err=newError('Thisshouldneverhappen');
}
next(err);//throwsiftheerrorwasset
})
Trivialloggingmiddleware:
module.context.use(function(req,res,next){
conststart=Date.now();
try{
next();
}finally{
console.log(`Handledrequestin${Date.now()-start}ms`);
}
});
M orecomplexexampleforheader-basedsessions:
constsessions=module.context.collection('sessions');
module.context.use({
register(endpoint){
endpoint.header('x-session-id',joi.string().optional(),'ThesessionID.');
returnfunction(req,res,next){
constsid=req.get('x-session-id');
if(sid){
try{
req.session=sessions.document(sid);
}catch(e){
deletereq.headers['x-session-id'];
}
}
next();
if(req.session){
if(req.session._rev){
sessions.replace(req.session,req.session);
res.set('x-session-id',req.session._key);
}else{
constmeta=sessions.save(req.session);
res.set('x-session-id',meta._key);
}
203
M iddleware
}
};
}
});
204
Request
Requestobjects
Therequestobjectspecifiesthefollowingproperties:
arangoUser: string|null
TheauthenticatedArangoDBusernameusedtomaketherequest.ThisvalueisonlysetifauthenticationisenabledinArangoDB
andtherequestsetan authorizationheaderArangoDBwasabletoverify.Youarestronglyencouragedtoimplementyourown
authenticationlogicforyourownservicesbutthispropertycanbeusefulifyouneedtointegratewithArangoDB'sown
authenticationmechanisms.
arangoVersion: number
Thenumericvalueofthe x-arango-versionheaderorthenumericversionoftheArangoDBserver(e.g. 30102forversion3.1.2)if
novalidheaderwasprovided.
baseUrl: string
Root-relativebaseURLoftheservice,i.e.theprefix "/_db/"followedbythevalueofdatabase.
body: any
Theprocessedandvalidatedrequestbodyforthecurrentroute.Ifnobodyhasbeendefinedforthecurrentroute,thevaluewillbe
identicaltorawBody.
FordetailsonhowrequestbodiescanbeprocessedandvalidatedbyFoxxseethebodymethodoftheendpointobject.
context: Context
Theservicecontextinwhichtherouterwasmounted(ratherthanthecontextinwhichtheroutewasdefined).
database: string
Thenameofthedatabaseinwhichtherequestisbeinghandled,e.g. "_system".
headers: object
Therawheadersobject.
FordetailsonhowrequestheaderscanbevalidatedbyFoxxseetheheadermethodoftheendpointobject.
hostname: string
Thehostname(domainname)indicatedintherequestheaders.
Defaultstothehostnameportion(i.e.excludingtheport)ofthe Hostheaderandfallsbacktothelisteningaddressoftheserver.
method: string
TheHTTPverbusedtomaketherequest,e.g. "GET".
originalUrl: string
Root-relativeURLoftherequest,i.e.pathfollowedbytherawqueryparameters,ifany.
path: string
Database-relativepathoftherequestURL(notincludingthequeryparameters).
pathParams: object
Anobjectmappingthenamesofpathparametersofthecurrentroutetotheirvalidatedvalues.
FordetailsonhowpathparameterscanbevalidatedbyFoxxseethepathParammethodoftheendpointobject.
port: number
Theportindicatedintherequestheaders.
205
Request
Defaultstotheportportion(i.e.excludingthehostname)ofthe Hostheaderandfallsbacktothelisteningportortheappropriate
defaultport( 443forHTTPSor 80forHTTP,dependingonsecure)iftheheaderonlyindicatesahostname.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettotheportportionofthe X-Forwarded-Hostheader(or
approriatedefaultport)ifpresent.
protocol: string
Theprotocolusedfortherequest.
Defaultsto "https"or "http"dependingonwhetherArangoDBisconfiguredtouseSSLornot.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothevalueofthe X-Forwarded-Protoheaderifpresent.
queryParams: object
Anobjectmappingthenamesofqueryparametersofthecurrentroutetotheirvalidatedvalues.
FordetailsonhowqueryparameterscanbevalidatedbyFoxxseethequeryParammethodoftheendpointobject.
rawBody: Buffer
Theraw,unparsed,unvalidatedrequestbodyasabuffer.
remoteAddress: string
TheIPoftheclientthatmadetherequest.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothefirstIPlistedinthe X-Forwarded-Forheaderif
present.
remoteAddresses: Array<string>
AlistcontainingtheIPaddressesusedtomaketherequest.
DefaultstothevalueofremoteAddresswrappedinanarray.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettothelistofIPsspecifiedinthe X-Forwarded-Forheader
ifpresent.
remotePort: number
Thelisteningportoftheclientthatmadetherequest.
Iftherequestwasmadeusingatrustedproxy(seetrustProxy),thisissettotheportspecifiedinthe X-Forwarded-Portheaderif
present.
secure: boolean
Whethertherequestwasmadeoverasecureconnection(i.e.HTTPS).
Thisissetto falsewhenprotocolis "http"and truewhenprotocolis "https".
suffix: string
Thetrailingpathrelativetothecurrentrouteifthecurrentrouteendsinawildcard(e.g. /something/*).
Note:StartingwithArangoDB3.2ispassedintotheserviceas-is,i.e.percentageescapesequenceslike %2Fwillnolongerbe
unescaped.Alsonotethatthesuffixmaycontainpathsegmentslike ..whichmayhavespecialmeaningifthesuffixisusedto
buildfilesystempaths.
trustProxy: boolean
Indicateswhethertherequestwasmadeusingatrustedproxy.Iftheoriginserver'saddresswasspecifiedintheArangoDB
configurationusing --frontend.trusted-proxyortheservice's trustProxysettingisenabled,thiswillbe true,otherwiseitwill
be false.
url: string
TheURLoftherequest.
206
Request
xhr: boolean
WhethertherequestindicatesitwasmadewithinabrowserusingAJAX.
Thisissetto trueifthe X-Requested-Withheaderispresentandisacase-insensitivematchforthevalue "xmlhttprequest".
NotethatthisvaluedoesnotguaranteewhethertherequestwasmadefrominsideabrowserorwhetherAJAXwasusedandis
merelyaconventionestablishedbyJavaScriptframeworkslikejQuery.
accepts
req.accepts(types):string|false
req.accepts(...types):string|false
req.acceptsCharsets(charsets):string|false
req.acceptsCharsets(...charsets):string|false
req.acceptsEncodings(encodings):string|false
req.acceptsEncodings(...encodings):string|false
req.acceptsLanguages(languages):string|false
req.acceptsLanguages(...languages):string|false
Thesemethodswrapthecorrespondingcontentnegotiationmethodsoftheacceptsmoduleforthecurrentrequest.
Examples
if(req.accepts(['json','html'])==='html'){
//ClientexplicitlyprefersHTMLoverJSON
res.write('<h1>ClientprefersHTML</h1>');
}else{
//OtherwisejustsendJSON
res.json({success:true});
}
cookie
req.cookie(name,options):string|null
Getsthevalueofacookiebyname.
Arguments
name: string
Nameofthecookie.
options: object(optional)
Anobjectwithanyofthefollowingproperties:
secret: string(optional)
Secretthatwasusedtosignthecookie.
Ifasecretisspecified,thecookie'ssignatureisexpectedtobepresentinasecondcookiewiththesamenameandthesuffix
.sig.Otherwisethesignature(ifpresent)willbeignored.
algorithm: string(Default: "sha256")
Algorithmthatwasusedtosignthecookie.
Ifastringispassedinsteadofanoptionsobjectitwillbeinterpretedasthesecretoption.
Returnsthevalueofthecookieor nullifthecookieisnotsetoritssignatureisinvalid.
207
Request
get/header
req.get(name):string
req.header(name):string
Getsthevalueofaheaderbyname.Youcanvalidaterequestheadersusingtheheadermethodoftheendpoint.
Arguments
name: string
Nameoftheheader.
Returnstheheadervalue.
is
req.is(types):string
req.is(...types):string
Thismethodwrapsthe(requestbody)contenttypedetectionmethodofthetype-ismoduleforthecurrentrequest.
Examples
consttype=req.is('html','application/xml','application/*+xml');
if(type===false){//nomatch
handleDefault(req.rawBody);
}elseif(type==='html'){
handleHtml(req.rawBody);
}else{//isXML
handleXml(req.rawBody);
}
json
req.json():any
AttemptstoparsetherawrequestbodyasJSONandreturnstheresult.
Itisgenerallymoreusefultodefinearequestbodyontheendpointandusethe req.bodypropertyinstead.
Returns undefinediftherequestbodyisempty.M aythrowa SyntaxErrorifthebodycouldnotbeparsed.
makeAbsolute
req.makeAbsolute(path,[query]):string
Resolvesthegivenpathrelativetothe req.context.service'smountpathtoafullURL.
Arguments
path: string
Thepathtoresovle.
query: string|object
AstringorobjectwithqueryparameterstoaddtotheURL.
ReturnstheformattedabsoluteURL.
params
req.param(name):any
208
Request
Arguments
Looksupaparameterbyname,preferring pathParamsover queryParams.
It'sprobablybetterstyletousethe req.pathParamsor req.queryParamsobjectsdirectly.
name: string
Nameoftheparameter.
Returnsthe(validated)valueoftheparameter.
range
req.range([size]):Ranges|number
Thismethodwrapstherangeheaderparsingmethodoftherange-parsermoduleforthecurrentrequest.
Arguments
size: number(Default: Infinity)
Lengthofthesatisfiablerange(e.g.numberofbytesinthefullresponse).Ifpresent,rangesexceedingthesizewillbeconsidered
unsatisfiable.
Returns undefinedifthe Rangeheaderisabsent, -2iftheheaderispresentbutmalformed, -1iftherangeisinvalid(e.g.start
offsetislargerthanendoffset)orunsatisfiableforthegivensize.
Otherwisereturnsanarrayofobjectswiththepropertiesstartandendvaluesforeachrange.Thearrayhasanadditionalpropertytype
indicatingtherequestrangetype.
Examples
console.log(req.headers.range);//"bytes=40-80"
constranges=req.range(100);
console.log(ranges);//[{start:40,end:80}]
console.log(ranges.type);//"bytes"
reverse
req.reverse(name,[params]):string
LooksuptheURLofanamedrouteforthegivenparameters.
Arguments
name: string
Nameoftheroutetolookup.
params: object(optional)
Anobjectcontainingvaluesforthe(pathorquery)parametersoftheroute.
ReturnstheURLoftherouteforthegivenparameters.
Examples
router.get('/items/:id',function(req,res){
/*...*/
},'getItemById');
router.post('/items',function(req,res){
//...
consturl=req.reverse('getItemById',{id:createdItem._key});
res.set('location',req.makeAbsolute(url));
});
209
Request
210
Response
Responseobjects
Theresponseobjectspecifiesthefollowingproperties:
body: Buffer|string
Responsebodyasastringorbuffer.Canbesetdirectlyorusingsomeoftheresponsemethods.
context: Context
Theservicecontextinwhichtherouterwasmounted(ratherthanthecontextinwhichtheroutewasdefined).
headers: object
Therawheadersobject.
statusCode: number
Statuscodeoftheresponse.Defaultsto 200(bodysetandnotanemptystringorbuffer)or 204(otherwise)ifnotchangedfrom
undefined.
attachment
res.attachment([filename]):this
Setsthe content-dispositionheadertoindicatetheresponseisadownloadablefilewiththegivenname.
Note:Thisdoesnotactuallymodifytheresponsebodyoraccessthefilesystem.Tosendafilefromthefilesystemseethe download
or sendFilemethods.
Arguments
filename: string(optional)
Nameofthedownloadablefileintheresponsebody.
Ifpresent,theextensionofthefilenamewillbeusedtosettheresponse content-typeifithasnotyetbeenset.
Returnstheresponseobject.
cookie
res.cookie(name,value,[options]):this
Setsacookiewiththegivenname.
Arguments
name: string
Nameofthecookie.
value: string
Valueofthecookie.
options: object(optional)
Anobjectwithanyofthefollowingproperties:
ttl: number(optional)
Timetoliveofthecookieinseconds.
algorithm: string(Default: "sha256")
Algorithmthatwillbeusedtosignthecookie.
211
Response
secret: string(optional)
Secretthatwillbeusedtosignthecookie.
Ifasecretisspecified,thecookie'ssignaturewillbestoredinasecondcookiewiththesameoptions,thesamenameandthe
suffix .sig.Otherwisenosignaturewillbeadded.
path: string(optional)
Pathforwhichthecookieshouldbeissued.
domain: string(optional)
Domainforwhichthecookieshouldbeissued.
secure: boolean(Default: false)
Whetherthecookieshouldbemarkedassecure(i.e.HTTPS/SSL-only).
httpOnly: boolean(Default: false)
WhetherthecookieshouldbemarkedasHTTP-only(ratherthanalsoexposingittoclient-sidecode).
Ifastringispassedinsteadofanoptionsobjectitwillbeinterpretedasthesecretoption.
Ifanumberispassedinsteadofanoptionsobjectitwillbeinterpretedasthettloption.
Returnstheresponseobject.
download
res.download(path,[filename]):this
Theequivalentofcalling res.attachment(filename).sendFile(path).
Arguments
path: string
Pathtothefileonthelocalfilesystemtobesentastheresponsebody.
filename: string(optional)
Filenametoindicateinthe content-dispositionheader.
Ifomittedthepathwillbeusedinstead.
Returnstheresponseobject.
getHeader
res.getHeader(name):string
Getsthevalueoftheheaderwiththegivenname.
Arguments
name: string
Nameoftheheadertoget.
Returnsthevalueoftheheaderor undefined.
json
res.json(data):this
SetstheresponsebodytotheJSONstringvalueofthegivendata.
212
Response
Arguments
data: any
Thedatatobeusedastheresponsebody.
Returnstheresponseobject.
redirect
res.redirect([status],path):this
Redirectstheresponsebysettingtheresponse locationheaderandstatuscode.
Arguments
status: number|string(optional)
Responsestatuscodetoset.
Ifthestatuscodeisthestringvalue "permanent"itwillbetreatedasthevalue 301.
Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.
Ifthestatuscodeisomittedbuttheresponsestatushasnotalreadybeenset,theresponsestatuswillbesetto 302.
path: string
URLtosetthe locationheaderto.
Returnstheresponseobject.
removeHeader
res.removeHeader(name):this
Removestheheaderwiththegivennamefromtheresponse.
Arguments
name: string
Nameoftheheadertoremove.
Returnstheresponseobject.
send
res.send(data,[type]):this
Setstheresponsebodytothegivendatawithrespecttotheresponsedefinitionfortheresponse'scurrentstatuscode.
Arguments
data: any
Thedatatobeusedastheresponsebody.Willbeconvertedaccordingtheresponsedefinitionfortheresponse'scurrentstatuscode
(or 200)inthefollowingway:
IfthedataisanArangoDBresultset,itwillbeconvertedtoanarrayfirst.
Iftheresponsedefinitionspecifiesamodelwitha forClientmethod,thatmethodwillbeappliedtothedatafirst.Ifthedataisan
arrayandtheresponsedefinitionhasthe multipleflagset,themethodwillbeappliedtoeachentryindividuallyinstead.
Finallythedatawillbeprocessedbytheresponsetypehandlertoconvertheresponsebodytoastringorbuffer.
type: string(Default: "auto")
213
Response
Content-typeoftheresponsebody.
Ifsetto "auto"thefirstM IM Etypespecifiedintheresponsedefinitionfortheresponse'scurrentstatuscode(or 200)willbe
usedinstead.
Ifsetto "auto"andnoresponsedefinitionexists,theM IM Etypewillbedeterminedthefollowingway:
IfthedataisabuffertheM IM Etypewillbesettobinary( application/octet-stream).
IfthedataisanobjecttheM IM EtypewillbesettoJSONandthedatawillbeconvertedtoaJSONstring.
OtherwisetheM IM EtypewillbesettoHTM Landthedatawillbeconvertedtoastring.
Returnstheresponseobject.
sendFile
res.sendFile(path,[options]):this
Sendsafilefromthelocalfilesystemastheresponsebody.
Arguments
path: string
Pathtothefileonthelocalfilesystemtobesentastheresponsebody.
Ifno content-typeheaderhasbeensetyet,theextensionofthefilenamewillbeusedtosetthevalueofthatheader.
options: object(optional)
Anobjectwithanyofthefollowingproperties:
lastModified: boolean(optional)
Ifsetto trueorifno last-modifiedheaderhasbeensetyetandthevalueisnotsetto falsethe last-modifiedheader
willbesettothemodificationdateofthefileinmilliseconds.
Returnstheresponseobject.
Examples
//Sendthefile"favicon.ico"fromthisservice'sfolder
res.sendFile(module.context.fileName('favicon.ico'));
sendStatus
res.sendStatus(status):this
Sendsaplaintextresponseforthegivenstatuscode.Theresponsestatuswillbesettothegivenstatuscode,theresponsebodywillbe
settothestatusmessagecorrespondingtothatstatuscode.
Arguments
status: number|string
Responsestatuscodetoset.
Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.
Returnstheresponseobject.
setHeader/set
res.setHeader(name,value):this
res.set(name,value):this
214
Response
res.set(headers):this
Setsthevalueoftheheaderwiththegivenname.
Arguments
name: string
Nameoftheheadertoset.
value: string
Valuetosettheheaderto.
headers: object
Headerobjectmappingheadernamestovalues.
Returnstheresponseobject.
status
res.status(status):this
Setstheresponsestatustothegivenstatuscode.
Arguments
status: number|string
Responsestatuscodetoset.
Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.
Returnstheresponseobject.
throw
res.throw(status,[reason],[options]):void
ThrowsanHTTPexceptionforthegivenstatus,whichwillbehandledbyFoxxtoservetheappropriateJSONerrorresponse.
Arguments
status: number|string
Responsestatuscodetoset.
Ifthestatuscodeisastringitwillbeconvertedtoanumericstatuscodeusingthestatusesmodulefirst.
Ifthestatuscodeisinthe500-range(500-599),itsstacktracewillalwaysbeloggedasifitwereanunhandledexception.
Ifdevelopmentmodeisenabled,theerror'sstacktracewillbeloggedasawarningifthestatuscodeisinthe400-range(400-499)or
asaregularmessageotherwise.
reason: string(optional)
M essagefortheexception.
Ifomitted,thestatusmessagecorrespondingtothestatuscodewillbeusedinstead.
options: object(optional)
Anobjectwithanyofthefollowingproperties:
cause: Error(optional)
Causeoftheexceptionthatwillbeloggedaspartoftheerror'sstacktrace(recursively,iftheexceptionalsohasa cause
propertyandsoon).
215
Response
extra: object(optional)
AdditionalpropertiesthatwillbeaddedtotheerrorresponsebodygeneratedbyFoxx.
Ifdevelopmentmodeisenabled,an exceptionpropertywillbeaddedtothisvaluecontainingtheerrormessageanda
stacktracepropertywillbeaddedcontaininganarraywitheachlineoftheerror'sstacktrace.
Ifanerrorispassedinsteadofanoptionsobjectitwillbeinterpretedasthecauseoption.Ifnoreasonwasprovidedtheerror's message
willbeusedasthereasoninstead.
Returnsnothing.
type
res.type([type]):string
Setstheresponsecontent-typetothegiventypeifprovidedorreturnsthepreviouslysetcontent-type.
Arguments
type: string(optional)
Content-typeoftheresponsebody.
Unlike res.set('content-type',type)fileextensionscanbeprovidedasvaluesandwillbetranslatedtothecorrespondingM IM E
type(e.g. jsonbecomes application/json).
Returnsthecontent-typeoftheresponsebody.
vary
res.vary(names):this
res.vary(...names):this
Thismethodwrapsthe varyheadermanipulationmethodofthevarymoduleforthecurrentresponse.
Thegivennameswillbeaddedtotheresponse's varyheaderifnotalreadypresent.
Returnstheresponseobject.
Examples
res.vary('user-agent');
res.vary('cookie');
res.vary('cookie');//duplicateswillbeignored
//--or-res.vary('user-agent','cookie');
//--or-res.vary(['user-agent','cookie']);
write
res.write(data):this
Appendsthegivendatatotheresponsebody.
Arguments
data: string|Buffer
Datatoappend.
216
Response
Ifthedataisabuffertheresponsebodywillbeconvertedtoabufferfirst.
Iftheresponsebodyisabufferthedatawillbeconvertedtoabufferfirst.
IfthedataisanobjectitwillbeconvertedtoaJSONstringfirst.
Ifthedataisanyothernon-stringvalueitwillbeconvertedtoastringfirst.
Returnstheresponseobject.
217
UsingGraphQL
UsingGraphQLinFoxx
constcreateGraphQLRouter=require('@arangodb/foxx/graphql');
Foxxbundlesthe graphql-syncmodule,whichisasynchronouswrapperfortheofficialJavaScriptGraphQLreferenceimplementation,
toallowwritingGraphQLschemasdirectlyinsideFoxx.
Additionallythe @arangodb/foxx/graphqlletsyoucreateroutersforservingGraphQLrequests,whichcloselymimicksthebehaviourof
the express-graphqlmodule.
Formoreinformationon graphql-syncseethe graphql-jsAPIreference(notethat graphql-syncalwaysusesrawvaluesinsteadof
wrappingtheminpromises).
ForanexampleofaGraphQLschemainFoxxthatresolvesfieldsusingthedatabaseseetheGraphQLexampleservice(alsoavailable
fromtheFoxxstore).
Examples
constgraphql=require('graphql-sync');
constgraphqlSchema=newgraphql.GraphQLSchema({
//...
});
//Mountingagraphqlendpointdirectlyinaservice:
module.context.use('/graphql',createGraphQLRouter({
schema:graphqlSchema,
graphiql:true
}));
//Orattheservice'srootURL:
module.context.use(createGraphQLRouter({
schema:graphqlSchema,
graphiql:true
}));
//Orinsideanexistingrouter:
router.get('/hello',function(req,res){
res.write('Helloworld!');
});
router.use('/graphql',createGraphQLRouter({
schema:graphqlSchema,
graphiql:true
}));
Note:ArangoDBaimsforstabilitywhichmeansbundleddependencieswillgenerallynotbeupdatedasquicklyastheirmaintainersmake
updatesavailableonGitHuborNPM .StartingwithArangoDB3.2,ifyouwanttouseanewerreleaseof graphql-syncthantheone
bundledwithyourtargetversionofArangoDB,youcanprovideyourownversionofthelibrarybypassingitviathe graphqloption:
constgraphql=require('graphql-sync');
constgraphqlSchema=newgraphql.Schema({
//...
});
module.context.use(createGraphQLRouter({
schema:graphqlSchema,
graphiql:true,
graphql:graphql
}))
ThismakessureFoxxusesthe graphql-syncmodulebundledinyourservice's node_modulesfolder(ifavailable)insteadofthebuilt-in
version.Ifyoufind graphql-syncitselflaggingbehindtheofficial graphqlmodule,consideropeninganissueonitsGitHub
repository.
Creatingarouter
createGraphQLRouter(options):Router
218
UsingGraphQL
ThisreturnsanewrouterobjectwithPOSTandGETroutesforservingGraphQLrequests.
Arguments
options: object
Anobjectwithanyofthefollowingproperties:
schema: GraphQLSchema
AGraphQLSchemaobjectfrom graphql-sync.
context: any(optional)
TheGraphQLcontextthatwillbepassedtothe graphql()functionfrom graphql-synctohandleGraphQLqueries.
rootValue: object(optional)
TheGraphQLrootvaluethatwillbepassedtothe graphql()functionfrom graphql-synctohandleGraphQLqueries.
pretty: boolean(Default: false)
If true,JSONresponseswillbepretty-printed.
formatError: Function(optional)
Afunctionthatwillbeusedtoformaterrorsproducedby graphql-sync.Ifomitted,the formatErrorfunctionfrom
graphql-syncwillbeusedinstead.
validationRules: Array<any>(optional)
AdditionalvalidationrulesqueriesmustsatisfyinadditiontothosedefinedintheGraphQLspec.
graphiql: boolean(Default: false)
If true,theGraphiQLexplorerwillbeservedwhenloadeddirectlyfromabrowser.
graphql: object(optional)
Ifyouneedtouseyourowncopyofthe graphql-syncmoduleinsteadoftheonebundledwithArangoDB,hereyoucanpass
itindirectly.
IfaGraphQLSchemaobjectispassedinsteadofanoptionsobjectitwillbeinterpretedastheschemaoption.
Generatedroutes
TherouterhandlesGETandPOSTrequeststoitsrootpathandacceptsthefollowingparameters,whichcanbeprovidedeitherasquery
parametersorasthePOSTrequestbody:
query: string
AGraphQLquerythatwillbeexecuted.
variables: object|string(optional)
AnobjectorastringcontainingaJSONobjectwithruntimevaluestouseforanyGraphQLqueryvariables.
operationName: string(optional)
Iftheprovided querycontainsmultiplenamedoperations,thisspecifieswhichoperationshouldbeexecuted.
raw: boolean(Default: false)
ForcesaJSONresponseevenifgraphiqlisenabledandtherequestwasmadeusingabrowser.
ThePOSTrequestbodycanbeprovidedasJSONorasquerystringusing application/x-www-form-urlencoded.Arequestbodypassed
as application/graphqlwillbeinterpretedasthe queryparameter.
219
Sessionsmiddleware
SessionMiddleware
constsessionMiddleware=require('@arangodb/foxx/sessions');
Thesessionmiddlewareaddsthe sessionand sessionStoragepropertiestotherequestobjectanddealswithserializingand
deserializingthesessionaswellasextractingsessionidentifiersfromincomingrequestsandinjectingthemintooutgoingresponses.
Examples
//Createasessionmiddleware
constsessions=sessionsMiddleware({
storage:module.context.collection('sessions'),
transport:['header','cookie']
});
//Firstenablethemiddlewareforthisservice
module.context.use(sessions);
//Nowmounttheroutersthatusethesession
constrouter=createRouter();
module.context.use(router);
router.get('/',function(req,res){
res.send(`Hello${req.session.uid||'anonymous'}!`);
},'hello');
router.post('/login',function(req,res){
req.session.uid=req.body;
req.sessionStorage.save(req.session);
res.redirect(req.reverse('hello'));
});
.body(['text/plain'],'Username');
Creatingasessionmiddleware
sessionMiddleware(options):Middleware
Createsasessionmiddleware.
Arguments
options: Object
Anobjectwiththefollowingproperties:
storage: Storage
Storagethatwillbeusedtopersistthesessions.
Thestorageisalsoexposedasthe sessionStorageonallrequestobjectsandasthe storagepropertyofthemiddleware.
IfastringorcollectionispassedinsteadofaStorage,itwillbeusedtocreateaCollectionStorage.
transport: Transport|Array<Transport>
Transportorarrayoftransportsthatwillbeusedtoextractthesessionidentifiersfromincomingrequestsandinjecttheminto
outgoingresponses.Whenattemptingtoextractasessionidentifier,thetransportswillbeusedintheorderspecifieduntila
matchisfound.Wheninjecting(orclearing)sessionidentifiers,alltransportswillbeinvoked.
Thetransportsarealsoexposedasthe transportpropertyofthemiddleware.
Ifthestring "cookie"ispassedinsteadofaTransport,theCookieTransportwillbeusedwiththedefaultsettingsinstead.
Ifthestring "header"ispassedinsteadofaTransport,theHeaderTransportwillbeusedwiththedefaultsettingsinstead.
autoCreate: boolean(Default: true)
Ifenabledthesessionstorage's newmethodwillbeinvokedtocreateanemptysessionwheneverthetransportfailedtoreturn
asessionfortheincomingrequest.Otherwisethesessionwillbeinitializedas null.
220
Sessionsmiddleware
Returnsthesessionmiddleware.
221
Sessionstorages
SessionStorages
Sessionstoragesareusedbythesessionsmiddlewaretopersistsessionsacrossrequests.Sessionstoragesmustimplementthe
fromClientand forClientmethodsandcanoptionallyimplementthe newmethod.
Thebuilt-insessionstoragesgenerallyprovidethefollowingattributes:
uid: string(Default: null)
Auniqueidentifierindicatingtheactiveuser.
created: number(Default: Date.now())
Thenumerictimestampofwhenthesessionwascreated.
data: any(Default: null)
Arbitrarydatatopersistedinthesession.
new
storage.new():Session
Generatesanewsessionobjectrepresentinganemptysession.Theemptysessionobjectshouldnotbepersistedunlessnecessary.The
returnvaluewillbeexposedbythemiddlewareasthe sessionpropertyoftherequestobjectifnosessionidentifierwasreturnedby
thesessiontransportsandauto-creationisnotexplicitlydisabledinthesessionmiddleware.
Examples
new(){
return{
uid:null,
created:Date.now(),
data:null
};
}
fromClient
storage.fromClient(sid):Session|null
Resolvesordeserializesasessionidentifiertoasessionobject.
Arguments
sid: string
Sessionidentifiertoresolveordeserialize.
Returnsasessionobjectrepresentingthesessionwiththegivensessionidentifierthatwillbeexposedbythemiddlewareasthe
sessionpropertyoftherequestobject.Thismethodwillonlybecalledifanyofthesessiontransportsreturnedasessionidentifier.If
thesessionidentifierisinvalidorexpired,themethodshouldreturna nullvaluetoindicatenomatchingsession.
Examples
fromClient(sid){
returndb._collection('sessions').firstExample({_key:sid});
}
forClient
storage.forClient(session):string|null
222
Sessionstorages
Derivesasessionidentifierfromthegivensessionobject.
Arguments
session: Session
Sessiontoderiveasessionidentifierfrom.
Returnsasessionidentifierforthesessionrepresentedbythegivensessionobject.Thismethodwillbecalledwiththe session
propertyoftherequestobjectunlessthatpropertyisempty(e.g. null).
Examples
forClient(session){
if(!session._key){
constmeta=db._collection('sessions').save(session);
returnmeta._key;
}
db._collection('sessions').replace(session._key,session);
returnsession._key;
}
223
Sessionstorages
CollectionSessionStorage
constcollectionStorage=require('@arangodb/foxx/sessions/storages/collection');
Thecollectionsessionstoragepersistssessionstoacollectioninthedatabase.
Creatingastorage
collectionStorage(options):Storage
CreatesaStoragethatcanbeusedinthesessionsmiddleware.
Arguments
options: Object
Anobjectwiththefollowingproperties:
collection: ArangoCollection
Thecollectionthatshouldbeusedtopersistthesessions.Ifastringispassedinsteadofacollectionitisassumedtobethe
fullyqualifiednameofacollectioninthecurrentdatabase.
ttl: number(Default: 60*60)
Thetimeinsecondssincethelastupdateuntilasessionwillbeconsideredexpired.
pruneExpired: boolean(Default: false)
Whetherexpiredsessionsshouldberemovedfromthecollectionwhentheyareaccessedinsteadofsimplybeingignored.
autoUpdate: boolean(Default: true)
Whethersessionsshouldbeupdatedinthecollectioneverytimetheyareaccessedtokeepthemfromexpiring.Disablingthis
optionwillimproveperformancebutmeansyouwillhavetotakecareofkeepingyoursessionsaliveyourself.
Ifastringorcollectionispassedinsteadofanoptionsobject,itwillbeinterpretedasthecollectionoption.
prune
storage.prune():Array<string>
Removesallexpiredsessionsfromthecollection.ThismethodshouldbecalledevenifthepruneExpiredoptionisenabledtocleanup
abandonedsessions.
Returnsanarrayofthekeysofallsessionsthatwereremoved.
save
storage.save(session):Session
Saves(replaces)thegivensessionobjectinthecollection.Thismethodneedstobeinvokedexplicitlyaftermakingchangestothesession
orthechangeswillnotbepersisted.Assignsanew _keytothesessionifitpreviouslydidnothaveone.
Arguments
session: Session
Asessionobject.
Returnsthemodifiedsession.
clear
224
Sessionstorages
storage.clear(session):boolean
Removesthesessionfromthecollection.Hasnoeffectifthesessionwasalreadyremovedorhasnotyetbeensavedtothecollection(i.e.
hasno _key).
Arguments
session: Session
Asessionobject.
Returns trueifthesessionwasremovedor falseifithadnoeffect.
225
Sessionstorages
JWTSessionStorage
constjwtStorage=require('@arangodb/foxx/sessions/storages/jwt');
TheJWTsessionstorageconvertssessionstoandfromJSONWebTokens.
Examples
//PassinasecuresecretfromtheFoxxconfiguration
constsecret=module.context.configuration.jwtSecret;
constsessions=sessionsMiddleware({
storage:jwtStorage(secret),
transport:'header'
});
module.context.use(sessions);
Creatingastorage
jwtStorage(options):Storage
CreatesaStoragethatcanbeusedinthesessionsmiddleware.
Note:whilethe"none"algorithm(i.e.nosignature)issupportedthisdummyalgorithmprovidesnosecurityandallowsclientstomake
arbitrarymodificationstothepayloadandshouldnotbeusedunlessyouarecertainyouspecificallyneedit.
Arguments
options: Object
Anobjectwiththefollowingproperties:
algorithm: string(Default: "HS512")
Thealgorithmtouseforsigningthetoken.
Supportedvalues:
"HS256"(HM AC-SHA256)
"HS384"(HM AC-SHA384)
"HS512"(HM AC-SHA512)
"none"(nosignature)
secret: string
Thesecrettouseforsigningthetoken.
Thisfieldisforbiddenwhenusingthe"none"algorithmbutrequiredotherwise.
ttl: number(Default: 3600)
Themaximumlifetimeofthetokeninseconds.Youmaywanttokeepthisshortasanewtokenisgeneratedoneveryrequest
allowingclientstorefreshtokensautomatically.
verify: boolean(Default: true)
Ifsetto falsethesignaturewillnotbeverifiedbutstillgenerated(unlessusingthe"none"algorithm).
maxExp: number(Default: Infinity)
LargestvaluethatwillbeacceptedinanincomingJWT exp(expiration)field.
Ifastringispassedinsteadofanoptionsobjectitwillbeinterpretedasthesecretoption.
226
Sessiontransports
SessionTransports
Sessiontransportsareusedbythesessionsmiddlewaretostoreandretrievesessionidentifiersinrequestsandresponses.Session
transportsmustimplementthe getand/or setmethodsandcanoptionallyimplementthe clearmethod.
get
transport.get(request):string|null
Retrievesasessionidentifierfromarequestobject.
Ifpresentthismethodwillautomaticallybeinvokedforeachtransportuntilatransportreturnsasessionidentifier.
Arguments
request: Request
Requestobjecttoextractasessionidentifierfrom.
Returnsthesessionidentifieror nullifthetransportcannotfindasessionidentifierintherequest.
Examples
get(req){
returnreq.get('x-session-id')||null;
}
set
transport.set(response,sid):void
Attachesasessionidentifiertoaresponseobject.
Ifpresentthismethodwillautomaticallybeinvokedattheendofarequestregardlessofwhetherthesessionwasmodifiedornot.
Arguments
response: Response
Responseobjecttoattachasessionidentifierto.
sid: string
Sessionidentifiertoattachtotheresponse.
Returnsnothing.
Examples
set(res){
res.set('x-session-id',value);
}
clear
transport.clear(response):void
Attachesapayloadindicatingthatthesessionhasbeenclearedtotheresponseobject.Thiscanbeusedtoclearasessioncookiewhenthe
sessionhasbeendestroyed(e.g.duringlogout).
Ifpresentthismethodwillautomaticallybeinvokedinsteadof setwhenthe req.sessionattributewasremovedbytheroutehandler.
Arguments
227
Sessiontransports
response: Response
Responseobjecttoremovethesessionidentifierfrom.
Returnsnothing.
228
Sessiontransports
CookieSessionTransport
constcookieTransport=require('@arangodb/foxx/sessions/transports/cookie');
Thecookietransportstoressessionidentifiersincookiesontherequestandresponseobject.
Examples
//PassinasecuresecretfromtheFoxxconfiguration
constsecret=module.context.configuration.cookieSecret;
constsessions=sessionsMiddleware({
storage:module.context.collection('sessions'),
transport:cookieTransport({
name:'FOXXSESSID',
ttl:60*60*24*7,//oneweekinseconds
algorithm:'sha256',
secret:secret
})
});
module.context.use(sessions);
Creatingatransport
cookieTransport([options]):Transport
CreatesaTransportthatcanbeusedinthesessionsmiddleware.
Arguments
options: Object(optional)
Anobjectwiththefollowingproperties:
name: string(Default: "sid")
Thenameofthecookie.
ttl: number(optional)
Cookielifetimeinseconds.NotethatthisdoesnotaffectthestorageTTL(i.e.howlongthesessionitselfisconsideredvalid),
justhowlongthecookieshouldbestoredbytheclient.
algorithm: string(optional)
Thealgorithmusedtosignandverifythecookie.Ifnoalgorithmisspecified,thecookiewillnotbesignedorverified.Seethe
cookiemethodontheresponseobject.
secret: string(optional)
Secrettouseforthesignedcookie.Willbeignoredifnoalgorithmisprovided.
path: string(optional)
Pathforwhichthecookieshouldbeissued.
domain: string(optional)
Domainforwhichthecookieshouldbeissued.
secure: boolean(Default: false)
Whetherthecookieshouldbemarkedassecure(i.e.HTTPS/SSL-only).
httpOnly: boolean(Default: false)
WhetherthecookieshouldbemarkedasHTTP-only(ratherthanalsoexposingittoclient-sidecode).
Ifastringispassedinsteadofanoptionsobject,itwillbeinterpretedasthenameoption.
229
Sessiontransports
230
Sessiontransports
HeaderSessionTransport
constheaderTransport=require('@arangodb/foxx/sessions/transports/header');
Theheadertransportstoressessionidentifiersinheadersontherequestandresponseobjects.
Examples
constsessions=sessionsMiddleware({
storage:module.context.collection('sessions'),
transport:headerTransport('X-FOXXSESSID')
});
module.context.use(sessions);
Creatingatransport
headerTransport([options]):Transport
CreatesaTransportthatcanbeusedinthesessionsmiddleware.
Arguments
options: Object(optional)
Anobjectwiththefollowingproperties:
name: string(Default: X-Session-Id)
Nameoftheheaderthatcontainsthesessionidentifier(notcasesensitive).
Ifastringispassedinsteadofanoptionsobject,itwillbeinterpretedasthenameoption.
231
Servingfiles
Staticfileassets
ThemostflexiblewaytoservefilesinyourFoxxserviceistosimplypassthemthroughinyourrouterusingthecontextobject's
fileNamemethodandtheresponseobject's sendFilemethod:
router.get('/some/filename.png',function(req,res){
constfilePath=module.context.fileName('some-local-filename.png');
res.sendFile(filePath);
});
Whileallowingforgreatercontrolofhowthefileshouldbesenttotheclientandwhoshouldbeabletoaccessit,doingthisforallyour
staticassetscangettedious.
AlternativelyyoucanspecifyfileassetsthatshouldbeservedbyyourFoxxservicedirectlyintheservicemanifestusingthe files
attribute:
"files":{
"/some/filename.png":{
"path":"some-local-filename.png",
"type":"image/png",
"gzip":false
},
"/favicon.ico":"bookmark.ico",
"/static":"my-assets-folder"
}
Eachentryinthe filesattributecanrepresenteitherasinglefileoradirectory.Whenservingentiredirectories,thekeyactsasaprefix
andrequeststothatprefixwillberesolvedwithinthegivendirectory.
Options
path: string
Therelativepathofthefileorfolderwithintheservice.
type: string(optional)
TheM IM Econtenttypeofthefile.Defaultstoanintelligentguessbasedonthefilename'sextension.
gzip: boolean(Default: false)
Ifsetto truethefilewillbeservedwithgzip-encodingifsupportedbytheclient.Thiscanbeusefulwhenservingtextfileslike
client-sideJavaScript,CSSorHTM L.
Ifastringisprovidedinsteadofanobject,itwillbeinterpretedasthepathoption.
232
Writingtests
Writingtests
FoxxprovidesoutoftheboxsupportforrunningtestsagainstaninstalledserviceusingtheM ochatestrunner.
TestfileshavefullaccesstotheservicecontextandallArangoDBAPIsbutlikescriptscannotdefineFoxxroutes.
Runningtests
Aninstalledservice'stestscanbeexecutedfromtheadministrativewebinterface:
1. Openthe"Services"tabofthewebinterface
2. Clickontheinstalledservicetobetested
3. Clickonthe"Settings"tab
4. Clickontheflaskiconinthetopright
5. Accepttheconfirmationdialog
Notethatrunningtestsinaproductiondatabaseisnotrecommendedandmayresultindatalossifthetestsaccessthedatabase.
Whenrunningaserviceindevelopmentmodespecialcareneedstobetakenasperformingrequeststotheservice'sownroutesaspartof
thetestsuitesmayresultintestsbeingexecutedwhilethedatabaseisinaninconsistentstate,leadingtounexpectedbehaviour.
Testfilepaths
InordertotellFoxxaboutfilescontainingtestsuites,oneormorepatternsneedtobespecifiedinthe testsoptionoftheservice
manifest:
{
"tests":[
"**/test_*.js",
"**/*_test.js"
]
}
Thesepatternscanbeeitherrelativefilepathsor"globstar"patternswhere
*matcheszeroormorecharactersinafilename
**matcheszeroormorenesteddirectories
Forexample,giventhefollowingdirectorystructure:
++test/
|++a/
||+-a1.js
||+-a2.js
||+-test.js
|+-b.js
|+-c.js
|+-d_test.js
+-e_test.js
+-test.js
Thefollowingpatternswouldmatchthefollowingfiles:
test.js:
test.js
test/*.js:
/test/b.js
/test/c.js
/test/d_test.js
233
Writingtests
test/**/*.js:
/test/a/a1.js
/test/a/a2.js
/test/a/test.js
/test/b.js
/test/c.js
/test/d_test.js
**/test.js:
/test/a/test.js
**/*test.js:
/test/a/test.js
/test/d_test.js
/e_test.js
/test.js
Evenifmultiplepatternsmatchthesamefilethetestsinthatfilewillonlyberunonce.
Theorderoftestsisalwaysdeterminedbythefilepaths,nottheorderinwhichtheyarematchedorspecifiedinthemanifest.
Teststructure
M ochatestsuitescanbedefinedusingoneofthreeinterfaces:BDD,TDDorExports.
TheQUnitinterfaceofM ochaisnotsupportedinArangoDB.
LikeallArangoDBcode,testcodeisalwayssynchronous.
BDDinterface
TheBDDinterfacedefinestestsuitesusingthe describefunctionandeachtestcaseisdefinedusingthe itfunction:
'usestrict';
constassert=require('assert');
consttrueThing=true;
describe('Truethings',()=>{
it('aretrue',()=>{
assert.equal(trueThing,true);
});
});
TheBDDinterfacealsooffersthealias contextfor describeand specifyfor it.
Testfixturescanbehandledusing beforeand afterforsuite-widefixturesand beforeEachand afterEachforper-testfixtures:
describe('Falsethings',()=>{
letfalseThing;
before(()=>{
falseThing=!true;
});
it('arefalse',()=>{
assert.equal(falseThing,false);
});
});
TDDinterface
TheTDDinterfacedefinestestsuitesusingthe suitefunctionandeachtestcaseisdefinedusingthe testfunction:
'usestrict';
constassert=require('assert');
consttrueThing=true;
suite('Truethings',()=>{
test('aretrue',()=>{
234
Writingtests
assert.equal(trueThing,true);
});
});
Testfixturescanbehandledusing suiteSetupand suiteTeardownforsuite-widefixturesand setupand teardownforper-test
fixtures:
suite('Falsethings',()=>{
letfalseThing;
suiteSetup(()=>{
falseThing=!true;
});
test('arefalse',()=>{
assert.equal(falseThing,false);
});
});
Exportsinterface
TheExportsinterfacedefinestestcasesasmethodsofplainobjectpropertiesofthe module.exportsobject:
'usestrict';
constassert=require('assert');
consttrueThing=true;
exports['Truethings']={
'aretrue':function(){
assert.equal(trueThing,true);
}
};
Thekeys before, after, beforeEachand afterEacharespecial-casedandbehavelikethecorrespondingfunctionsintheBDD
interface:
letfalseThing;
exports['Falsethings']={
before(){
falseThing=false;
},
'arefalse':function(){
assert.equal(falseThing,false);
}
};
Assertions
ArangoDBprovidestwobundledmodulestodefineassertions:
assertcorrespondstotheNode.js assertmodule,providinglow-levelassertionsthatcanoptionallyspecifyanerrormessage.
chaiisthepopularChaiAssertionLibrary,providingbothBDDandTDDstyleassertionsusingafamiliarsyntax.
235
CrossOrigin
Cross-OriginResourceSharing(CORS)
TouseCORSinyourFoxxservicesyoufirstneedtoconfigureArangoDBforCORS.Asof3.2Foxxwillthenautomaticallywhitelistall
responseheadersastheyareused.
IfyouwantmorecontroloverthewhitelistorareusinganolderversionofArangoDByoucansetthefollowingresponseheadersinyour
requesthandler:
access-control-expose-headers:acomma-separatedlistofresponseheaders.Thisdefaultstoalistofallheaderstheresponseis
actuallyusing(butnotincludingany access-controlheaders).
access-control-allow-credentials:canbesetto "false"toforbidexposingcookies.Thedefaultvaluedependsonwhether
ArangoDBtruststheorigin.Seethenoteson http.trusted-origin.
NotethatitisnotpossibletooverridetheseheadersfortheCORSpreflightresponse.Itisthereforenotpossibletoacceptcredentialsor
cookiesonlyforindividualroutes,servicesordatabases.TheoriginneedstobetrustedaccordingtothegeneralArangoDBconfiguration
(seeabove).
236
Scriptsandqueuedjobs
Foxxscriptsandqueuedjobs
Foxxletsyoudefinescriptsthatcanbeexecutedaspartoftheinstallationandremovalprocess,invokedmanuallyorscheduledtorunat
alatertimeusingthejobqueue.
Toregisteryourscript,justadda scriptssectiontoyourservicemanifest:
{
...
"scripts":{
"setup":"scripts/setup.js",
"send-mail":"scripts/send-mail.js"
}
...
}
Thescriptsyoudefineinyourservicemanifestcanbeinvokedfromthewebinterfaceintheservice'ssettingspagewiththeScripts
dropdown.
Youcanalsousethescriptsasqueuedjobs:
'usestrict';
constqueues=require('@arangodb/foxx/queues');
queues.get('default').push(
{mount:'/my-service-mount-point',name:'send-mail'},
{to:'user@example.com',body:'Hello'}
);
Scriptargumentsandreturnvalues
Ifthescriptwasinvokedwithanyarguments,youcanaccessthemusingthe module.context.argvarray.
Toreturndatafromyourscript,youcanassignthedatato module.exportsasusual.Pleasenotethatthisdatawillbeconvertedto
JSON.
Anyerrorsraisedbythescriptwillbehandleddependingonhowthescriptwasinvoked:
ifthescriptwasinvokedfromtheHTTPAPI(e.g.usingthewebinterface),itwillreturnanerrorresponseusingtheexception's
statusCodepropertyifspecifiedor500.
ifthescriptwasinvokedfromaFoxxjobqueue,thejob'sfailurecounterwillbeincrementedandthejobwillberescheduledor
markedasfailedifnoattemptsremain.
Examples
Let'ssayyouwanttodefineascriptthattakestwonumericvaluesandreturnstheresultofmultiplyingthem:
'usestrict';
constassert=require('assert');
constargv=module.context.argv;
assert.equal(argv.length,2,'Expectedexactlytwoarguments');
assert.equal(typeofargv[0],'number','Expectedfirstargumenttobeanumber');
assert.equal(typeofargv[1],'number','Expectedsecondargumenttobeanumber');
module.exports=argv[0]*argv[1];
LifecycleScripts
Foxxrecognizeslifecyclescriptsiftheyaredefinedandwillinvokethemduringtheinstallation,updateandremovalprocessofthe
serviceifyouwant.
237
Scriptsandqueuedjobs
Thefollowingscriptsarecurrentlyrecognizedaslifecyclescriptsbytheirname: "setup"and "teardown".
SetupScript
ThesetupscriptwillbeexecutedwithoutargumentsduringtheinstallationofyourFoxxservice.
Thesetupscriptmaybeexecutedmorethanonceandshouldthereforebetreatedasreentrant.Runningthesamesetupscriptagain
shouldnotresultinanyerrorsorduplicatedata.
Thesetupscriptistypicallyusedtocreatecollectionsyourserviceneedsorinsertseeddatalikeinitialadministrativeuseraccountsand
soon.
Examples
'usestrict';
constdb=require('@arangodb').db;
consttextsCollectionName=module.context.collectionName('texts');
//`textsCollectionName`isnowtheprefixednameofthisservice's"texts"collection.
//e.g."example_texts"iftheservicehasbeenmountedat`/example`
if(db._collection(textsCollectionName)===null){
constcollection=db._create(textsCollectionName);
collection.save({text:'entry1fromcollectiontexts'});
collection.save({text:'entry2fromcollectiontexts'});
collection.save({text:'entry3fromcollectiontexts'});
}else{
console.debug(`collection${texts}alreadyexists.Leavingituntouched.`);
}
TeardownScript
TheteardownscriptwillbeexecutedwithoutargumentsduringtheremovalofyourFoxxservice.
Itcanalsooptionallybeexecutedbeforeupgradinganservice.
Thisscripttypicallyremovesthecollectionsand/ordocumentscreatedbyyourservice'ssetupscript.
Examples
'usestrict';
constdb=require('@arangodb').db;
consttextsCollection=module.context.collection('texts');
if(textsCollection){
textsCollection.drop();
}
Queues
constqueues=require('@arangodb/foxx/queues')
Foxxallowsdefiningjobqueuesthatletyouperformsloworexpensiveactionsasynchronously.Thesequeuescanbeusedtosendemails,callexternalAPIsorperformotheractionsthatyoudonotwanttoperformdirectlyorwanttoretryonfailure.
enableordisabletheFoxxqueuesfeature --foxx.queuesflagIftrue,theFoxxqueueswillbeavailableandjobsinthequeueswillbe
executedasynchronously.Thedefaultistrue.Whensetto falsethequeuemanagerwillbedisabledandanyjobsarepreventedfrom
beingprocessed,whichmayreduceCPUloadabit.PleasenotethatFoxxjobqueuesaredatabase-specific.Queuesandjobsarealways
relativetothedatabaseinwhichtheyarecreatedoraccessed.
pollintervalforFoxxqueues --foxx.queues-poll-intervalvalueThepollintervalfortheFoxxqueuesmanager.Thevalueisspecifiedin
seconds.LowervalueswillmeanmoreimmediateandmorefrequentFoxxqueuejobexecution,butwillmakethequeuethreadwakeup
andquerythequeuesmoreoften.Whensettoalowvalue,thequeuethreadmightcauseCPUload.Thedefaultis1second.IfFoxx
238
Scriptsandqueuedjobs
queuesarenotusedmuch,thenthisvaluemaybeincreasedtomakethequeuesthreadwakeupless.Forthelow-levelfunctionalitysee
thechapteronthetaskmanagementmodule.
Creatingorupdatingaqueue
queues.create(name,[maxWorkers]):Queue
Returnsthequeueforthegivenname.Ifthequeuedoesnotexist,anewqueuewiththegivennamewillbecreated.Ifaqueuewiththe
givennamealreadyexistsandmaxWorkersisset,thequeue'smaximumnumberofworkerswillbeupdated.Thequeuewillbecreatedin
thecurrentdatabase.
Arguments
name: string
Nameofthequeuetocreate.
maxWorkers: number(Default: 1)
Themaximumnumberofworkers.
Examples
//Createaqueuewiththedefaultnumberofworkers(i.e.one)
constqueue1=queues.create("my-queue");
//Createaqueuewithagivennumberofworkers
constqueue2=queues.create("another-queue",2);
//Updatethenumberofworkersofanexistingqueue
constqueue3=queues.create("my-queue",10);
//queue1andqueue3refertothesamequeue
assertEqual(queue1,queue3);
Fetchinganexistingqueue
queues.get(name):Queue
Returnsthequeueforthegivenname.Ifthequeuedoesnotexistanexceptionisthrowninstead.
Thequeuewillbelookedupinthecurrentdatabase.
Arguments
name: string
Nameofthequeuetofetch.
Examples
Ifthequeuedoesnotyetexistanexceptionisthrown:
queues.get("some-queue");
//Error:Queuedoesnotexist:some-queue
//at...
Otherwisethequeuewillbereturned:
constqueue1=queues.create("some-queue");
constqueue2=queues.get("some-queue");
assertEqual(queue1,queue2);
Deletingaqueue
queues.delete(name):boolean
Returns trueifthequeuewasdeletedsuccessfully.Ifthequeuedidnotexist,itreturns falseinstead.Thequeuewillbelookedup
anddeletedinthecurrentdatabase.
239
Scriptsandqueuedjobs
Whenaqueueisdeleted,jobsonthatqueuewillnolongerbeexecuted.
Deletingaqueuewillnotdeleteanyjobsonthatqueue.
Arguments
name: string
Nameofthequeuetodelete.
Examples
constqueue=queues.create("my-queue");
queues.delete("my-queue");//true
queues.delete("my-queue");//false
Addingajobtoaqueue
queue.push(script,data,[opts]):string
Thejobwillbeaddedtothespecifiedqueueinthecurrentdatabase.
Returnsthejobid.
Arguments
script: object
Ajobtypedefinition,consistingofanobjectwiththefollowingproperties:
name: string
Nameofthescriptthatwillbeinvoked.
mount: string
M ountpathoftheservicethatdefinesthescript.
backOff: Function|number(Default: 1000)
Eitherafunctionthattakesthenumberoftimesthejobhasfailedbeforeasinputandreturnsthenumberofmillisecondsto
waitbeforetryingthejobagain,orthedelaytobeusedtocalculateanexponentialback-off,or 0fornodelay.
maxFailures: number|Infinity(Default: 0):
Numberoftimesasinglerunofajobwillbere-triedbeforeitismarkedas "failed".Anegativevalueor Infinitymeans
thatthejobwillbere-triedonfailureindefinitely.
schema: Schema(optional)
Schematovalidateajob'sdataagainstbeforeenqueuingthejob.
preprocess: Function(optional)
Functiontopre-processajob's(validated)databeforeserializingitinthequeue.
repeatTimes: Function(Default: 0)
Ifsettoapositivenumber,thejobwillberepeatedthismanytimes(notcountingrecoverywhenusingmaxFailures).Ifsetto
anegativenumberor Infinity,thejobwillberepeatedindefinitely.Ifsetto 0thejobwillnotberepeated.
repeatUntil: number|Date(optional)
Ifthejobissettoautomaticallyrepeat,thiscanbesettoatimestampinmilliseconds(or Dateinstance)afterwhichthejob
willnolongerrepeat.Settingthisvaluetozero,anegativevalueor Infinityhasnoeffect.
repeatDelay: number(Default: 0)
Ifthejobissettoautomaticallyrepeat,thiscanbesettoanon-negativevaluetosetthenumberofmillisecondsforwhichthe
jobwillbedelayedbeforeitisstartedagain.
240
Scriptsandqueuedjobs
data: any
Jobdataofthejob;mustbeserializabletoJSON.
opts: object(optional)
Objectwithanyofthefollowingproperties:
success: Function(optional)
Functiontobecalledafterthejobhasbeencompletedsuccessfully.
failure: Function(optional)
Functiontobecalledafterthejobhasfailedtoomanytimes.
delayUntil: number|Date(Default: Date.now())
Timestampinmilliseconds(or Dateinstance)untilwhichtheexecutionofthejobshouldbedelayed.
backOff: Function|number(Default: 1000)
Seescript.backOff.
maxFailures: number|Infinity(Default: 0):
Seescript.maxFailures.
repeatTimes: Function(Default: 0)
Seescript.repeatTimes.
repeatUntil: number|Date(optional)
Seescript.repeatUntil.
repeatDelay: number(Default: 0)
Seescript.repeatDelay.
NotethatifyoupassafunctionforthebackOffcalculation,successcallbackorfailurecallbackoptionsthefunctionwillbeserializedto
thedatabaseasastringandthereforemustnotrelyonanyexternalscopeorexternalvariables.
Whenthejobissettoautomaticallyrepeat,thefailurecallbackwillonlybeexecutedwhenarunofthejobhasfailedmorethan
maxFailurestimes.NotethatifthejobfailsandmaxFailuresisset,itwillberescheduledaccordingtothebackOffuntilithaseither
failedtoomanytimesorcompletedsuccessfullybeforebeingscheduledaccordingtotherepeatDelayagain.Recoveryattemptsby
maxFailuresdonotcounttowardsrepeatTimes.
Thesuccessandfailurecallbacksreceivethefollowingarguments:
result: any
Thereturnvalueofthescriptforthecurrentrunofthejob.
jobData: any
Thedatapassedtothismethod.
job: object
ArangoDBdocumentrepresentingthejob'scurrentstate.
Examples
Let'ssaywehaveanservicemountedat /mailerthatprovidesascriptcalled send-mail:
'usestrict';
constqueues=require('@arangodb/foxx/queues');
constqueue=queues.create('my-queue');
queue.push(
{mount:'/mailer',name:'send-mail'},
{to:'hello@example.com',body:'Helloworld'}
);
Thiswillnotwork,because logwasdefinedoutsidethecallbackfunction(thecallbackmustbeserializabletoastring):
//WARNING:THISDOESNOTWORK!
241
Scriptsandqueuedjobs
'usestrict';
constqueues=require('@arangodb/foxx/queues');
constqueue=queues.create('my-queue');
constlog=require('console').log;//outsidethecallback'sfunctionscope
queue.push(
{mount:'/mailer',name:'send-mail'},
{to:'hello@example.com',body:'Helloworld'},
{success:function(){
log('Yay!');//throws'logisnotdefined'
}}
);
Here'sanexampleofajobthatwillbeexecutedevery5secondsuntiltomorrow:
'usestrict';
constqueues=require('@arangodb/foxx').queues;
constqueue=queues.create('my-queue');
queue.push(
{mount:'/mailer',name:'send-mail'},
{to:'hello@example.com',body:'Helloworld'},
{
repeatTimes:Infinity,
repeatUntil:Date.now()+(24*60*60*1000),
repeatDelay:5*1000
}
);
Fetchingajobfromthequeue
queue.get(jobId):Job
Createsaproxyobjectrepresentingajobwiththegivenjobid.
Thejobwillbelookedupinthespecifiedqueueinthecurrentdatabase.
ReturnsthejobforthegivenjobId.Propertiesofthejobobjectwillbefetchedwhenevertheyarereferencedandcannotbemodified.
Arguments
jobId: string
Theidofthejobtocreateaproxyobjectfor.
Examples
constjobId=queue.push({mount:'/logger',name:'log'},'HelloWorld!');
constjob=queue.get(jobId);
assertEqual(job.id,jobId);
Deletingajobfromthequeue
queue.delete(jobId):boolean
Deletesajobwiththegivenjobid.Thejobwillbelookedupanddeletedinthespecifiedqueueinthecurrentdatabase.
Arguments
jobId: string
Theidofthejobtodelete.
Returns trueifthejobwasdeletedsuccessfully.Ifthejobdidnotexistitreturns falseinstead.
Fetchinganarrayofjobsinaqueue
Examples
constlogScript={mount:'/logger',name:'log'};
queue.push(logScript,'HelloWorld!',{delayUntil:Date.now()+50});
242
Scriptsandqueuedjobs
assertEqual(queue.pending(logScript).length,1);
//50mslater...
assertEqual(queue.pending(logScript).length,0);
assertEqual(queue.progress(logScript).length,1);
//evenlater...
assertEqual(queue.progress(logScript).length,0);
assertEqual(queue.complete(logScript).length,1);
Fetchinganarrayofpendingjobsinaqueue
queue.pending([script]):Array<string>
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus "pending",optionallyfilteredbythegivenjobtype.Thejobswill
belookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
script: object(optional)
Anobjectwiththefollowingproperties:
name: string
Nameofthescript.
mount: string
M ountpathoftheservicedefiningthescript.
Fetchinganarrayofjobsthatarecurrentlyinprogress
queue.progress([script])
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus "progress",optionallyfilteredbythegivenjobtype.Thejobs
willbelookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
script: object(optional)
Anobjectwiththefollowingproperties:
name: string
Nameofthescript.
mount: string
M ountpathoftheservicedefiningthescript.
Fetchinganarrayofcompletedjobsinaqueue
queue.complete([script]):Array<string>
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus "complete",optionallyfilteredbythegivenjobtype.Thejobs
willbelookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
script: object(optional)
Anobjectwiththefollowingproperties:
name: string
Nameofthescript.
mount: string
M ountpathoftheservicedefiningthescript.
Fetchinganarrayoffailedjobsinaqueue
queue.failed([script]):Array<string>
243
Scriptsandqueuedjobs
Returnsanarrayofjobidsofjobsinthegivenqueuewiththestatus "failed",optionallyfilteredbythegivenjobtype.Thejobswill
belookedupinthespecifiedqueueinthecurrentdatabase.
Arguments
script: object(optional)
Anobjectwiththefollowingproperties:
name: string
Nameofthescript.
mount: string
M ountpathoftheservicedefiningthescript.
Fetchinganarrayofalljobsinaqueue
queue.all([script]):Array<string>
Returnsanarrayofjobidsofalljobsinthegivenqueue,optionallyfilteredbythegivenjobtype.Thejobswillbelookedupinthe
specifiedqueueinthecurrentdatabase.
Arguments
script: object(optional)
Anobjectwiththefollowingproperties:
name: string
Nameofthescript.
mount: string
M ountpathoftheservicedefiningthescript.
Abortingajob
job.abort():void
Abortsanon-completedjob.
Setsajob'sstatusto "failed"ifitisnotalready "complete",withoutcallingthejob'sonFailurecallback.
244
M igrating2.xservices
Migrating2.xservicesto3.0
WhenmigratingservicesfromolderversionsofArangoDBitisgenerallyrecommendedyoumakesuretheyworkinlegacycompatibility
mode,whichcanalsoserveasastop-gapsolution.
ThischapteroutlinesthemajordifferencesintheFoxxAPIbetweenArangoDB2.8andArangoDB3.0.
Generalchanges
The consoleobjectinlaterversionsofArangoDB2.ximplementedaspecialFoxxconsoleAPIandwouldoptionallylogmessagestoa
collection.ArangoDB3.0restorestheoriginalbehaviourwhere consoleisthesameobjectavailablefromtheconsolemodule.
245
M igratingfrompre-2.8
Migratingfrompre-2.8
WhenmigratingfromaversionolderthanArangoDB2.8pleasenotethatstartingwithArangoDB2.8thebehaviourofthe require
functionmorecloselymimicsthebehaviourobservedinNode.jsandmodulebundlersforbrowsers,e.g.:
Inafile /routes/examples.js(relativetotherootfolderoftheservice):
require('./my-module')willbeattemptedtoberesolvedinthefollowingorder:
1.
/routes/my-module(relativetoserviceroot)
2.
/routes/my-module.js(relativetoserviceroot)
3.
/routes/my-module.json(relativetoserviceroot)
4.
/routes/my-module/index.js(relativetoserviceroot)
5.
/routes/my-module/index.json(relativetoserviceroot)
require('lodash')willbeattemptedtoberesolvedinthefollowingorder:
1.
/routes/node_modules/lodash(relativetoserviceroot)
2.
/node_modules/lodash(relativetoserviceroot)
3. ArangoDBmodule lodash
4. Nodecompatibilitymodule lodash
5. BundledNPM module lodash
require('/abs/path')willbeattemptedtoberesolvedinthefollowingorder:
1.
/abs/path(relativetofilesystemroot)
2.
/abs/path.js(relativetofilesystemroot)
3.
/abs/path.json(relativetofilesystemroot)
4.
/abs/path/index.js(relativetofilesystemroot)
5.
/abs/path/index.json(relativetofilesystemroot)
ThisbehaviourisincompatiblewiththesourcecodegeneratedbytheFoxxgeneratorinthewebinterfacebeforeArangoDB2.8.
Note:The org/arangodbmoduleisaliasedtothenewname @arangodbinArangoDB3.0.0andthe @arangodbmodulewasaliasedto
theoldname org/arangodbinArangoDB2.8.0.Eitheronewillworkin2.8and3.0butoutsideoflegacyservicesyoushoulduse
@arangodbgoingforward.
Foxxqueue
InArangoDB2.6FoxxintroducedanewwaytodefinequeuedjobsusingFoxxscriptstoreplacethefunction-basedjobtypedefinitions
whichwerecausingproblemswhenrestartingtheserver.Thefunction-basedjobshavebeenremovedin2.7andarenolongersupported
atall.
CoffeeScript
ArangoDB3.0nolongerprovidesbuilt-insupportforCoffeeScriptsourcefiles,eveninlegacycompatibilitymode.Ifyouwanttousean
alternativelanguagelikeCoffeeScript,makesuretopre-compiletherawsourcefilestoJavaScriptandusethecompiledJavaScriptfilesin
theservice.
Therequestmodule
The @arangodb/requestmodulewhenusedwiththe jsonoptionpreviouslyoverwrotethestringinthe bodypropertyofthe
responseobjectoftheresponsewiththeparsedJSONbody.In2.8thiswaschangedsotheparsedJSONbodyisaddedasthe json
propertyoftheresponseobjectinadditiontooverwritingthe bodyproperty.In3.0andlater(includinglegacycompatibilitymode)the
bodypropertyisnolongeroverwrittenandmustusethe jsonpropertyinstead.Notethatthisonlyaffectscodeusingthe json
optionwhenmakingtherequest.
246
M igratingfrompre-2.8
BundledNPMmodules
ThebundledNPM moduleshavebeenupgradedandmayincludebackwards-incompatiblechanges,especiallytheAPIof joihas
changedseveraltimes.Ifindoubtyoushouldbundleyourownversionsofthesemodulestoensurespecificversionswillbeused.
Theutilitymodule lodashisnowavailableandshouldbeusedinsteadof underscore,butbothmoduleswillcontinuetobeprovided.
247
manifest.json
Manifest
M anyofthefieldsthatwererequiredinArangoDB2.xarenowoptionalandcanbesafelyomitted.
ToavoidcompatibilityproblemswithfutureversionsofArangoDByoushouldalwaysspecifythe enginesfield,e.g.:
{
"engines":{
"arangodb":"^3.0.0"
}
}
Controllers&exports
PreviouslyFoxxdistinguishedbetween exportsand controllers,eachofwhichcouldbespecifiedasanobject.InArangoDB3.0
thesehavebeenmergedintoasingle mainfieldspecifyinganentryfile.
Theeasiestwaytomigrateservicesusingmultipleexportsand/orcontrollersistocreateaseparateentryfilethatimportsthesefiles:
Old(manifest.json):
{
"exports":{
"doodads":"doodads.js",
"dingbats":"dingbats.js"
},
"controllers":{
"/doodads":"routes/doodads.js",
"/dingbats":"routes/dingbats.js",
"/":"routes/root.js"
}
}
New(manifest.json):
{
"main":"index.js"
}
New(index.js):
'usestrict';
module.context.use('/doodads',require('./routes/doodads'));
module.context.use('/dingbats',require('./routes/dingbats'));
module.context.use('/',require('./routes/root'));
module.exports={
doodads:require('./doodads'),
dingbats:require('./dingbats')
};
Indexredirect
Ifyoupreviouslydidnotdefinethe defaultDocumentfield,pleasenotethatinArangoDB3.0thefieldwillnolongerdefaulttothevalue
index.htmlwhenomitted:
Old:
{
//nodefaultDocument
}
248
manifest.json
New:
{
"defaultDocument":"index.html"
}
Thisalsomeansitisnolongernecessarytospecifythe defaultDocumentfieldwithanemptyvaluetopreventtheredirectandbeableto
serverequestsatthe /(root)pathofthemountpoint:
Old:
{
"defaultDocument":""
}
New:
{
//nodefaultDocument
}
Assets
The assetsfieldisnolongersupportedinArangoDB3.0outsideoflegacycompatibilitymode.
Ifyoupreviouslyusedthefieldtoserveindividualfilesas-isyoucansimplyusethe filesfieldinstead:
Old:
{
"assets":{
"client.js":{
"files":["assets/client.js"],
"contentType":"application/javascript"
}
}
}
New:
{
"files":{
"client.js":{
"path":"assets/client.js",
"type":"application/javascript"
}
}
}
Ifyoureliedonbeingabletospecifymultiplefilesthatshouldbeconcatenated,youwillhavetousebuildtoolsoutsideofArangoDBto
preparethesefilesaccordingly.
Rootelement
The rootElementfieldisnolongersupportedandhasbeenremovedentirely.
IfyourcontrollersreliedonthisfieldbeingavailableyouneedtoadjustyourschemasandroutestobeabletohandlethefullJSON
structureofincomingdocuments.
Systemservices
249
manifest.json
The isSystemfieldisnolongersupported.ThepresenceorabsenceofthefieldhadnoeffectinmostrecentversionsofArangoDB2.x
andhasnowbeenremovedentirely.
250
applicationContext
Theapplicationcontext
Theglobal applicationContextvariableavailableinFoxxmoduleshasbeenreplacedwiththe contextattributeofthe module
variable.Forconsistencyitisnowreferredtoastheservicecontextthroughoutthisdocumentation.
SomemethodsoftheservicecontexthavechangedinArangoDB3.0:
fileName()nowbehaveslike path()didinArangoDB2.x
path()hasbeenremoved(use fileName()instead)
foxxFileName()hasbeenremoved(use fileName()instead)
Additionallythe versionand nameattributeshavebeenremovedandcannowonlybeaccessedviathe manifestattribute(as
manifest.versionand manifest.name).Notethatthecorrespondingmanifestfieldsarenowoptionalandmaybeomitted.
The optionsattributehasalsobeenremovedasitshouldbeconsideredanimplementationdetail.Youshouldinsteadaccessthe
dependenciesand configurationattributesdirectly.
Theinternal _prefixattribute(whichwasanaliasfor basePath)andtheinternal commentand clearCommentsmethods(whichwere
usedbythemagicaldocumentationcommentsinArangoDB2.x)havealsobeenremoved.
Theinternal _serviceattribute(whichprovidesaccesstotheserviceitself)hasbeenrenamedto service.
251
RepositoriesandM odels
Repositoriesandmodels
PreviouslyFoxxwasheavilybuiltaroundtheconceptofrepositoriesandmodels,whichprovidedcomplexbutrarelynecessary
abstractionsontopofArangoDBcollectionsanddocuments.InArangoDB3.0thesehavebeenremovedentirely.
Repositoriesvscollections
RepositoriesmostlywrappedmethodsthatalreadyexistedonArangoDBcollectionobjectsandprimarilydealtwithconvertingbetween
plainArangoDBdocumentsandFoxxmodelinstances.InArangoDB3.0youcansimplyusethesecollectionsdirectlyandtreat
documentsasplainJavaScriptobjects.
Old:
'usestrict';
constFoxx=require('org/arangodb/foxx');
constmyRepo=newFoxx.Repository(
applicationContext.collection('myCollection'),
{model:Foxx.Model}
);
//...
constmodels=myRepo.byExample({color:'green'});
res.json(models.map(function(model){
returnmodel.forClient();
}));
New:
'usestrict';
constmyDocs=module.context.collection('myCollection');
//...
constdocs=myDocs.byExample({color:'green'});
res.json(docs);
Schemavalidation
ThemainpurposeofmodelsinArangoDB2.xwastovalidateincomingdatausingjoischemas.InmorerecentversionsofArangoDB2.x
itwasalreadypossibletopasstheseschemasdirectlyinmostplaceswhereamodelwasexpectedasanargument.Theonlydifferenceis
thatschemasshouldnowbeconsideredthedefault.
IfyoupreviouslyreliedontheautomaticvalidationofFoxxmodelinstanceswhensettingattributesorinstantiatingmodelsfrom
untrusteddata,youcansimplyusetheschema's validatemethoddirectly.
Old:
'usestrict';
constjoi=require('joi');
constmySchema={
name:joi.string().required(),
size:joi.number().required()
};
constFoxx=require('org/arangodb/foxx');
constMyModel=Foxx.Model.extend({schema:mySchema});
//...
constmodel=newMyModel(req.json());
if(!model.isValid){
res.status(400);
252
RepositoriesandM odels
res.write('Badrequest');
return;
}
New:
'usestrict';
constjoi=require('joi');
//Notethisisnowwrappedinajoi.object()
constmySchema=joi.object({
name:joi.string().required(),
size:joi.number().required()
}).required();
//...
constresult=mySchema.validate(req.body);
if(result.errors){
res.status(400);
res.write('Badrequest');
return;
}
Migratingmodels
Whilemostusecasesformodelscannowbereplacedwithplainjoischemas,thereisstilltheconceptofa"model"inFoxxinArangoDB
3.0althoughitisquitedifferentfromFoxxmodelsinArangoDB2.x.
AmodelinFoxxnowreferstoaplainJavaScriptobjectwithanoptional schemaattributeandtheoptionalmethods forClientand
fromClient.M odelscanbeusedinsteadofplainjoischemastodefinerequestandresponsebodiesbuttherearenomodel"instances"in
ArangoDB3.0.
Old:
'usestrict';
const_=require('underscore');
constjoi=require('joi');
constFoxx=require('org/arangodb/foxx');
constMyModel=Foxx.Model.extend({
schema:{
name:joi.string().required(),
size:joi.number().required()
},
forClient(){
return_.omit(this.attributes,['_key','_id','_rev']);
}
});
//...
ctrl.get(/*...*/)
.bodyParam('body',{type:MyModel});
New:
'usestrict';
const_=require('lodash');
constjoi=require('joi');
constMyModel={
schema:joi.object({
name:joi.string().required(),
size:joi.number().required()
}).required(),
forClient(data){
return_.omit(data,['_key','_id','_rev']);
}
};
253
RepositoriesandM odels
//...
router.get(/*...*/)
.body(MyModel);
Triggers
Whensaving,updating,replacingordeletingmodelsinArangoDB2.xusingtherepositorymethodstherepositoryandmodelwouldfire
eventsthatcouldbesubscribedtoinordertoperformside-effects.
Notethatevenin2.xtheseeventswouldnotfirewhenusingqueriesormanipulatingdocumentsinanyotherwaythanusingthespecific
repositorymethodsthatoperatedonindividualdocuments.
ThisbehaviourisnolongeravailableinArangoDB3.0butcanbeemulatedbyusingan EventEmitterdirectlyifitisnotpossibleto
solvetheproblemdifferently:
Old:
'usestrict';
constFoxx=require('org/arangodb/foxx');
constMyModel=Foxx.Model.extend({
//...
},{
afterRemove(){
console.log(this.get('name'),'wasremoved');
}
});
//...
constmodel=myRepo.firstExample({name:'myName'});
myRepo.remove(model);
//->"myNamewasremovedsuccessfully"
New:
'usestrict';
constEventEmitter=require('events');
constemitter=newEventEmitter();
emitter.on('afterRemove',function(doc){
console.log(doc.name,'wasremoved');
});
//...
constdoc=myDocs.firstExample({name:'myName'});
myDocs.remove(doc);
emitter.emit('afterRemove',doc);
//->"myNamewasremovedsuccessfully"
Orsimply:
'usestrict';
functionafterRemove(doc){
console.log(doc.name,'wasremoved');
}
//...
constdoc=myDocs.firstExample({name:'myName'});
myDocs.remove(doc);
afterRemove(doc);
//->"myNamewasremovedsuccessfully"
254
Controllers
Controllersvsrouters
FoxxControllershavebeenreplacedwithrouters.Thisismorethanacosmeticchangeastherearesignificantdifferencesinbehaviour:
Controllerswereautomaticallymountedwhenthefiledefiningthemwasexecuted.Routersneedtobeexplicitlymountedusingthe
module.context.usemethod.Routerscanalsobeexported,importedandevennested.Thismakesiteasiertosplitupcomplexrouting
treesacrossmultiplefiles.
Old:
'usestrict';
constFoxx=require('org/arangodb/foxx');
constctrl=newFoxx.Controller(applicationContext);
ctrl.get('/hello',function(req,res){
//...
});
New:
'usestrict';
constcreateRouter=require('org/arangodb/foxx/router');
constrouter=createRouter();
//Ifyouareimportingthisfilefromyourentryfile("main"):
module.exports=router;
//Otherwise:module.context.use(router);
router.get('/hello',function(req,res){
//...
});
Somegeneralchangesinbehaviourthatmighttripyouup:
WhenspecifyingpathparameterswithschemasFoxxwillnowignoretherouteiftheschemadoesnotmatch(i.e. /hello/foxxwill
nolongermatch /hello/:numif numspecifiesaschemathatdoesn'tmatchthevalue "foxx").Withcontrollersthiscould
previouslyresultinusersseeinga400(badrequest)errorwhentheyshouldinsteadbeserveda404(notfound)response.
WhenarequestismadewithanHTTPverbnotsupportedbyanendpoint,Foxxwillnowrespondwitha405(methodnotallowed)
errorwithanappropriate AllowedheaderlistingthesupportedHTTPverbsforthatendpoint.
FoxxwillnolongerparseyourJSDoccommentstogenerateroutedocumentation(usethe summaryand descriptionmethodsof
theendpointinstead).
The apiDocumentationmethodnowlivesontheservicecontextandbehavesslightlydifferently.
Thereisnorouterequivalentforthe activateAuthenticationand activateSessionsmethods.Insteadyoushouldusethesession
middleware(seethesectiononsessionsbelow).
Thereisno delaliasforthe deletemethodonrouters.IthasalwaysbeensafetousekeywordsasmethodnamesinFoxx,so
theuseofthisaliaswasalreadydiscouragedbefore.
The allRoutesproxyisnolotavailableonroutersbutcaneasilybereplacedwithmiddlewareorchildrouters.
255
Controllers
Therequestcontext
Whendefiningarouteonacontrollerthecontrollerwouldreturnanobjectcalledrequestcontext.Routersreturnasimilarobjectcalled
endpoint.Routersalsoreturnendpointswhenmountingchildroutersormiddleware,asdoesthe usemethodoftheservicecontext.
ThemaindifferencesbetweenthenewendpointsandtheobjectsreturnedbycontrollersinpreviousversionsofArangoDBare:
bodyParamisnowsimplycalled body;itisnolongerneccessaryorpossibletogivethebodyanameandtherequestbodywill
notshowupintherequestparameters.It'salsopossibletospecifyaM IM Etype
body, queryParamand pathParamnowtakepositionargumentsinsteadofanobject.Forspecificsseetheendpoint
documentation.
notesisnowcalled descriptionandtakesasinglestringargument.
onlyIfand onlyIfAuthenticatedarenolongeravailable;theycanbeemulatedwithmiddlewareifnecessary:
Old:
ctrl.get(/*...*/)
.onlyIf(function(req){
if(!req.user){
thrownewError('Notauthenticated!');
}
});
New:
router.use(function(req,res,next){
if(!req.arangoUser){
res.throw(403,'Notauthenticated!');
}
next();
});
router.get(/*...*/);
256
Controllers
Errorhandling
The errorResponsemethodprovidedbycontrollerrequestcontextshasnoequivalentinrouterendpoints.Ifyouwanttohandle
specificerrortypeswithspecificstatuscodesyouneedtocatchthemexplicitly,eitherintherouteorinamiddleware:
Old:
ctrl.get('/puppies',function(req,res){
//Exceptionisthrownhere
})
.errorResponse(TooManyPuppiesError,400,'Somethingwentwrong!');
New:
ctrl.get('/puppies',function(req,res){
try{
//Exceptionisthrownhere
}catch(e){
if(!(einstanceofTooManyPuppiesError)){
throwe;
}
res.throw(400,'Somethingwentwrong!');
}
})
//The"error"methodmerelydocumentsthemeaning
//ofthestatuscodeandhasnoothereffect.
.error(400,'Throwniftherearetoomanypuppies.');
Notethaterrorscreatedwith http-errorsarestillhandledbyFoxxintelligently.Infact res.throwisjustahelpermethodforcreating
andthrowingtheseerrors.
257
Controllers
Before,afterandaround
The before, afterand aroundmethodscaneasilybereplacedbymiddleware:
Old:
letstart;
ctrl.before(function(req,res){
start=Date.now();
});
ctrl.after(function(req,res){
console.log('Requesthandledin',(Date.now()-start),'ms');
});
New:
router.use(function(req,res,next){
letstart=Date.now();
next();
console.log('Requesthandledin',(Date.now()-start),'ms');
});
Notethatunlike aroundmiddlewarereceivesthe nextfunctionasthethirdargument(the"opts"argumenthasnoequivalent).
258
Controllers
Requestobjects
Thenamesofsomeattributesoftherequestobjecthavebeenadjustedtomorecloselyalignwiththoseofthecorrespondingmethodson
theendpointobjectsandestablishedconventionsinotherJavaScriptframeworks:
req.urlParametersisnowcalled req.pathParams
req.parametersisnowcalled req.queryParams
req.params()isnowcalled req.param()
req.requestTypeisnowcalled req.method
req.compatibilityisnowcalled req.arangoVersion
req.userisnowcalled req.arangoUser
Someattributeshavebeenremovedorchanged:
req.cookieshasbeenremovedentirely(use req.cookie(name))
req.requestBodyhasbeenremovedentirely(seebelow)
req.suffixisnowastringratherthananarray
Additionallythe req.serverand req.clientattributesarenolongeravailable.Theinformationisnowexposedinawaythatcan
(optionally)transparentlyhandleproxyforwardingheaders:
req.hostnamedefaultsto req.server.address
req.portdefaultsto req.server.port
req.remoteAddressdefaultsto client.address
req.remotePortdefaultsto client.port
Finally,the req.cookiemethodnowtakesthe signedoptionsdirectly.
Old:
constsid=req.cookie('sid',{
signed:{
secret:'keyboardcat',
algorithm:'sha256'
}
});
New:
constsid=req.cookie('sid',{
secret:'keyboardcat',
algorithm:'sha256'
});
Requestbodies
The req.bodyisnolongeramethodandnolongerautomaticallyparsesJSONrequestbodiesunlessarequestbodywasdefined.The
req.rawBodynowcorrespondstothe req.rawBodyBufferofArangoDB2.xandisalsonolongeramethod.
Old:
ctrl.post('/',function(req,res){
constdata=req.body();
//...
});
259
Controllers
New:
router.post('/',function(req,res){
constdata=req.body;
//...
})
.body(['json']);
Orsimply:
constjoi=require('joi');
router.post('/',function(req,res){
constdata=req.body;
//...
})
.body(joi.object().optional());
Multipartrequests
The req.requestPartsmethodhasbeenremovedentirely.Ifyouneedtoacceptmultipartrequestbodies,youcansimplydefinethe
requestbodyusingamultipartM IM Etypelike multipart/form-data:
Old:
ctrl.post('/',function(req,res){
constparts=req.requestParts();
//...
});
New:
router.post('/',function(req,res){
constparts=req.body;
//...
})
.body(['multipart/form-data']);
260
Controllers
Responseobjects
TheresponseobjecthasalotofnewmethodsinArangoDB3.0butotherwiseremainssimilartotheresponseobjectofprevious
versions:
The res.sendmethodbehavesverydifferentlyfromhowthemethodwiththesamenamebehavedinArangoDB2.x:theconversion
nowtakestheresponsebodydefinitionoftherouteintoaccount.Thereisanewmethod res.writethatimplementstheoldbehaviour.
Notethatconsecutivecallsto res.writewillappendtotheresponsebodyratherthanreplacingitlike res.send.
The res.contentTypepropertyisalsonolongeravailable.IfyouwanttosettheM IM Etypeoftheresponsebodytoanexplicitvalue
youshouldsetthe content-typeheaderinstead:
Old:
res.contentType='application/json';
res.body=JSON.stringify(results);
New:
res.set('content-type','application/json');
res.body=JSON.stringify(results);
Orsimply:
//setsthecontenttypetoJSON
//ifithasnotalreadybeenset
res.json(results);
The res.cookiemethodnowtakesthe signedoptionsaspartoftheregularoptionsobject.
Old:
res.cookie('sid','abcdef',{
ttl:60*60,
signed:{
secret:'keyboardcat',
algorithm:'sha256'
}
});
New:
res.cookie('sid','abcdef',{
ttl:60*60,
secret:'keyboardcat',
algorithm:'sha256'
});
261
Controllers
Dependencyinjection
Thereisnoequivalentofthe addInjectormethodavailableinArangoDB2.xcontrollers.M ostusecasescanbesolvedbysimplyusing
plainvariablesbutifyouneedsomethingmoreflexibleyoucanalsousemiddleware:
Old:
ctrl.addInjector('magicNumber',function(){
returnMath.random();
});
ctrl.get('/',function(req,res,injected){
res.json(injected.magicNumber);
});
New:
functionmagicMiddleware(name){
return{
register(){
letmagic;
returnfunction(req,res,next){
if(!magic){
magic=Math.random();
}
req[name]=magic;
next();
};
}
};
}
router.use(magicMiddleware('magicNumber'));
router.get('/',function(req,res){
res.json(req.magicNumber);
});
Orsimply:
constmagicNumber=Math.random();
router.get('/',function(req,res){
res.json(magicNumber);
});
262
Sessions
Sessions
The ctrl.activateSessionsmethodandtherelated util-sessions-localFoxxservicehavebeenreplacedwiththeFoxxsessions
middleware.Itisnolongerpossibletousethebuilt-insessionstoragebutyoucansimplypassinanydocumentcollectiondirectly.
Old:
constlocalSessions=applicationContext.dependencies.localSessions;
constsessionStorage=localSessions.sessionStorage;
ctrl.activateSessions({
sessionStorage:sessionStorage,
cookie:{secret:'keyboardcat'}
});
ctrl.destroySession('/logout',function(req,res){
res.json({message:'Goodbye!'});
});
New:
constsessionMiddleware=require('@arangodb/foxx/sessions');
constcookieTransport=require('@arangodb/foxx/sessions/transports/cookie');
router.use(sessionMiddleware({
storage:module.context.collection('sessions'),
transport:cookieTransport('keyboardcat')
}));
router.post('/logout',function(req,res){
req.sessionStorage.clear(req.session);
res.json({message:'Goodbye!'});
});
263
AuthandOAuth2
AuthandOAuth2
The util-simple-authand util-oauth2FoxxserviceshavebeenreplacedwiththeFoxxauthandFoxxOAuth2modules.Itisno
longernecessarytoinstalltheseservicesasdependenciesinordertousethefunctionality.
Old:
'usestrict';
constauth=applicationContext.dependencies.simpleAuth;
//...
constvalid=auth.verifyPassword(authData,password);
New:
'usestrict';
constcreateAuth=require('@arangodb/foxx/auth');
constauth=createAuth();//Usedefaultconfiguration
//...
constvalid=auth.verifyPassword(authData,password);
264
FoxxQueries
Foxxqueries
The createQuerymethodhasbeenremoved.ItcanbetriviallyreplacedwithplainJavaScriptfunctionsanddirectcallstothe
db._querymethod:
Old:
'usestrict';
constFoxx=require('org/arangodb/foxx');
constquery=Foxx.createQuery({
query:'FORuIN_usersSORTu.userASCRETURNu[@propName]',
params:['propName'],
transform:function(results,uppercase){
return(
uppercase
?results[0].toUpperCase()
:results[0].toLowerCase()
);
}
});
query('user',true);
New:
'usestrict';
constdb=require('@arangodb').db;
constaql=require('@arangodb').aql;
functionquery(propName,uppercase){
constresults=db._query(aql`
FORuIN_users
SORTu.userASC
RETURNu[${propName}]
`);
return(
uppercase
?results[0].toUpperCase()
:results[0].toLowerCase()
);
}
query('user',true);
265
Legacycompatibilitymode
Legacycompatibilitymodefor2.8services
ArangoDB3continuestosupportFoxxserviceswrittenforArangoDB2.8byrunningtheminaspeciallegacycompatibilitymodethat
providesaccesstosomeofthemodulesandAPIsnolongerprovidedin3.0andbeyond.
Note:Legacycompatibilitymodeisstrictlyintendedasatemporarystopgapsolutionforsupportingexistingserviceswhileupgrading
toArangoDB3.0andshouldnotbeconsideredapermanentfeatureofArangoDBorFoxx.
Inordertomarkanexistingserviceasalegacyservice,justmakesurethefollowingattributeisdefinedintheservicemanifest:
"engines":{
"arangodb":"^2.8.0"
}
ThissemanticversionrangedenotesthattheserviceisknowntoworkwithArangoDB2.8.0andsupportsallnewerversionsof
ArangoDBuptobutnotincluding3.0.0(noranydevelopmentversionof3.0.0andgreater).
Anysimilarversionrangethedoesnotinclude3.0.0orgreaterwillhavethesameeffect(e.g. ^2.5.0willalsotriggerthelegacy
compatibilitymode,aswill 1.2.3,but >=2.8.0willnotasitindicatescompatibilitywithallversionsgreaterorequal2.8.0,notjust
thosewithinthe2.xversionrange).
Featuressupportedinlegacycompatibilitymode
Legacycompatibilitymodesupportstheoldmanifestformat,specifically:
mainisignored
controllerswillbemountedasin2.8
exportswillbeexecutedasin2.8
Additionallythe isSystemattributewillbeignoredifpresentbutdoesnotresultinawarninginlegacycompatibilitymode.
TheFoxxconsoleisavailableasthe consolepseudo-globalvariable(shadowingtheglobalconsoleobject).
Theservicecontextisavailableasthe applicationContextpseudo-globalvariableinthe controllers, exports, scriptsand tests
asin2.8.Thefollowingadditionalpropertiesareavailableontheservicecontextinlegacycompatibilitymode:
path()isanaliasfor3.x fileName()(using path.jointobuildfilepaths)
fileName()behavesasin2.x(using fs.safeJointobuildfilepaths)
foxxFileName()isanaliasfor2.x fileName
versionexposestheservicemanifest's versionattribute
nameexposestheservicemanifest's nameattribute
optionsexposestheservice'srawoptions
Thefollowingmethodsareremovedontheservicecontextinlegacycompatibilitymode:
use()--use @arangodb/foxx/controllerinstead
apiDocumentation()--use controller.apiDocumentation()instead
registerType()--notsupportedinlegacycompatibilitymode
Thefollowingmodulesthathavebeenremovedorreplacedin3.0.0areavailableinlegacycompatibilitymode:
@arangodb/foxx/authentication
@arangodb/foxx/console
@arangodb/foxx/controller
@arangodb/foxx/model
@arangodb/foxx/query
@arangodb/foxx/repository
@arangodb/foxx/schema
@arangodb/foxx/sessions
@arangodb/foxx/template_middleware
266
Legacycompatibilitymode
The @arangodb/foxxmodulealsoprovidesthesameexportsasin2.8,namely:
Controllerfrom @arangodb/foxx/controller
createQueryfrom @arangodb/foxx/query
Modelfrom @arangodb/foxx/model
Repositoryfrom @arangodb/foxx/repository
toJSONSchemafrom @arangodb/foxx/schema
getExportsand requireAppfrom @arangodb/foxx/manager
queuesfrom @arangodb/foxx/queues
Anyfeaturenotsupportedin2.8willalsonotworkinlegacycompatibilitymode.WhenmigratingfromanolderversionofArangoDBit
isagoodideatomigratetoArangoDB2.8firstforaneasierupgradepath.
AdditionallypleasenotethedifferenceslaidoutinthechapterMigratingfrompre-2.8inthemigrationguide.
267
Usermanagement
Usermanagement
Foxxdoesnotprovideanyusermanagementoutoftheboxbutitisveryeasytorollyourownsolution:
Thesessionmiddlewareprovidesmechanismsforaddingsessionlogictoyourservice,usinge.g.acollectionorJSONWebTokensto
storethesessionsbetweenrequests.
Theauthmoduleprovidesutilitiesforbasicpasswordverificationandhashing.
Thefollowingexampleservicedemonstrateshowusermanagementcanbeimplementedusingthesebasicbuildingblocks.
Settingupthecollections
Let'ssaywewanttostoresessionsandusersincollections.Wecanusethesetupscripttomakesurethesecollectionsarecreatedbefore
theserviceismounted.
Firstaddasetupscripttoyourmanifestifitisn'talreadydefined:
"scripts":{
"setup":"scripts/setup.js"
}
Thencreatethesetupscriptwiththefollowingcontent:
'usestrict';
constdb=require('@arangodb').db;
constsessions=module.context.collectionName('sessions');
constusers=module.context.collectionName('users');
if(!db._collection(sessions)){
db._createDocumentCollection(sessions);
}
if(!db._collection(users)){
db._createDocumentCollection(users);
}
db._collection(users).ensureIndex({
type:'hash',
fields:['username'],
unique:true
});
Creatingtherouter
Thefollowingmainfiledemonstratesbasicusermanagement:
'usestrict';
constjoi=require('joi');
constcreateAuth=require('@arangodb/foxx/auth');
constcreateRouter=require('@arangodb/foxx/router');
constsessionsMiddleware=require('@arangodb/foxx/sessions');
constauth=createAuth();
constrouter=createRouter();
constusers=module.context.collection('users');
constsessions=sessionsMiddleware({
storage:module.context.collection('sessions'),
transport:'cookie'
});
module.context.use(sessions);
module.context.use(router);
router.get('/whoami',function(req,res){
268
Usermanagement
try{
constuser=users.document(req.session.uid);
res.send({username:user.username});
}catch(e){
res.send({username:null});
}
})
.description('Returnsthecurrentlyactiveusername.');
router.post('/login',function(req,res){
//Thismayreturnauserobjectornull
constuser=users.firstExample({
username:req.body.username
});
constvalid=auth.verify(
//Pretendtovalidateevenifnouserwasfound
user?user.authData:{},
req.body.password
);
if(!valid)res.throw('unauthorized');
//Logtheuserin
req.session.uid=user._key;
req.sessionStorage.save(req.session);
res.send({sucess:true});
})
.body(joi.object({
username:joi.string().required(),
password:joi.string().required()
}).required(),'Credentials')
.description('Logsaregistereduserin.');
router.post('/logout',function(req,res){
if(req.session.uid){
req.session.uid=null;
req.sessionStorage.save(req.session);
}
res.send({success:true});
})
.description('Logsthecurrentuserout.');
router.post('/signup',function(req,res){
constuser=req.body;
try{
//Createanauthenticationhash
user.authData=auth.create(user.password);
deleteuser.password;
constmeta=users.save(user);
Object.assign(user,meta);
}catch(e){
//Failedtosavetheuser
//We'llassumetheUniqueConstrainthasbeenviolated
res.throw('badrequest','Usernamealreadytaken',e);
}
//Logtheuserin
req.session.uid=user._key;
req.sessionStorage.save(req.session);
res.send({success:true});
})
.body(joi.object({
username:joi.string().required(),
password:joi.string().required()
}).required(),'Credentials')
.description('Createsanewuserandlogsthemin.');
269
Relatedmodules
Relatedmodules
ThesearesomeofthemodulesoutsideofFoxxyouwillfindusefulwhenwritingFoxxservices.
AdditionallytherearemodulesprovidingsomelevelofcompatibilitywithNode.jsaswellasanumberofbundledNPM modules(like
lodashandjoi).FormoreinformationonthesemodulesseetheJavaScriptmodulesappendix.
The @arangodbmodule
require('@arangodb')
ThismoduleprovidesaccesstovariousArangoDBinternalsaswellasthreeofthemostimportantexportsnecessarytoworkwiththe
databaseinFoxx: db, aqland errors.
YoucanfindafulldescriptionofthismoduleintheArangoDBmoduleappendix.
The @arangodb/requestmodule
require('@arangodb/request')
ThismoduleprovidesafunctionformakingHTTPrequeststoexternalservices.NotethatwhilethisallowscommunicatingwiththirdpartyservicesitmayaffectdatabaseperformancebyblockingFoxxrequestsasArangoDBwaitsfortheremoteservicetorespond.If
youroutinelymakerequeststoslowexternalservicesandarenotdirectlyinterestedintheresponseitisprobablyabetterideato
delegatetheactualrequest/responsecycletoagatewayservicerunningoutsideArangoDB.
Youcanfindafulldescriptionofthismoduleintherequestmoduleappendix.
The @arangodb/general-graphmodule
require('@arangodb/general-graph')
ThismoduleprovidesaccesstoArangoDBgraphdefinitionsandvariouslow-levelgraphoperationsinJavaScript.Formorecomplex
queriesitisprobablybettertouseAQLbutthismodulecanbeusefulinyoursetupandteardownscriptstocreateanddestroygraph
definitions.
Formoreinformationseethechapteronthegeneralgraphmodule.
270
Authentication
Authentication
constcreateAuth=require('@arangodb/foxx/auth');
Authenticatorsallowimplementingbasicpasswordmechanismusingsimplebuilt-inhashingfunctions.
ForafullexampleofsessionswithauthenticationandregistrationseetheexampleinthechapteronUserM anagement.
Creatinganauthenticator
createAuth([options]):Authenticator
Createsanauthenticator.
Arguments
options: Object(optional)
Anobjectwiththefollowingproperties:
method: string(Default: "sha256")
Thehashingalgorithmtousetocreatepasswordhashes.Theauthenticatorwillbeabletoverifypasswordsagainsthashes
usinganysupportedhashingalgorithm.Thisonlyaffectsnewhashescreatedbytheauthenticator.
Supportedvalues:
"md5"
"sha1"
"sha224"
"sha256"
"sha384"
"sha512"
saltLength: number(Default: 16)
Lengthofthesaltsthatwillbegeneratedforpasswordhashes.
Returnsanauthenticator.
Creatingauthenticationdataobjects
auth.create(password):AuthData
Createsanauthenticationdataobjectforthegivenpasswordwiththefollowingproperties:
method: string
Themethodusedtogeneratethehash.
salt: string
Arandomsaltusedtogeneratethishash.
hash: string
Thehashstringitself.
Arguments
password: string
Apasswordtohash.
Returnstheauthenticationdataobject.
271
Authentication
Validatingpasswordsagainstauthenticationdataobjects
auth.verify([hash,[password]]):boolean
Verifiesthegivenpasswordagainstthegivenhashusingaconstanttimestringcomparison.
Arguments
hash: AuthData(optional)
Aauthenticationdataobjectgeneratedwiththecreatemethod.
password: string(optional)
Apasswordtoverifyagainstthehash.
Returns trueifthehashmatchesthegivenpassword.Returns falseotherwise.
272
OAuth1.0a
OAuth1.0a
constcreateOAuth1Client=require('@arangodb/foxx/oauth1');
TheOAuth1moduleprovidesabstractionsoverOAuth1.0aproviderslikeTwitter,XINGandTumblr.
Examples
Thefollowingextendstheusermanagementexample:
constrouter=createRouter();
constoauth1=createOAuth1Client({
//We'lluseTwitterforthisexample
requestTokenEndpoint:'https://api.twitter.com/oauth/request_token',
authEndpoint:'https://api.twitter.com/oauth/authorize',
accessTokenEndpoint:'https://api.twitter.com/oauth/access_token',
activeUserEndpoint:'https://api.twitter.com/1.1/account/verify_credentials.json',
clientId:'keyboardcat',
clientSecret:'keyboardcat'
});
module.context.use('/oauth1',router);
//Seetheusermanagementexampleforsettingupthe
//sessionsandusersobjectsusedinthisexample
router.use(sessions);
router.post('/auth',function(req,res){
consturl=req.reverse('oauth1_callback');
constoauth_callback=req.makeAbsolute(url);
constrequestToken=oauth1.fetchRequestToken(oauth_callback);
if(requestToken.oauth_callback_confirmed!=='true'){
res.throw(500,'CouldnotfetchOAuthrequesttoken');
}
//Setrequesttokencookieforfiveminutes
res.cookie('oauth1_request_token',requestToken.oauth_token,{ttl:60*5});
//Redirecttotheprovider'sauthorizationURL
res.redirect(303,oauth1.getAuthUrl(requestToken.oauth_token));
});
router.get('/auth',function(req,res){
//MakesureCSRFcookiematchestheURL
constexpectedToken=req.cookie('oauth1_request_token');
if(!expectedToken||req.queryParams.oauth_token!==expectedToken){
res.throw(400,'CSRFmismatch.');
}
constauthData=oauth1.exchangeRequestToken(
req.queryParams.oauth_token,
req.queryParams.oauth_verifier
);
consttwitterToken=authData.oauth_token;
consttwitterSecret=authData.oauth_token_secret;
//Fetchtheactiveuser'sprofileinfo
constprofile=oauth1.fetchActiveUser(twitterToken,twitterSecret);
consttwitterId=profile.screen_name;
//TrytofindanexistinguserwiththeuserID
//(thisrequirestheuserscollection)
letuser=users.firstExample({twitterId});
if(user){
//UpdatethetwitterTokenifithaschanged
if(
user.twitterToken!==twitterToken||
user.twitterSecret!==twitterSecret
){
users.update(user,{twitterToken,twitterSecret});
}
}else{
//Createanewuserdocument
user={
username:`twitter:${twitterId}`,
twitterId,
twitterToken
273
OAuth1.0a
}
constmeta=users.save(user);
Object.assign(user,meta);
}
//Logtheuserin(thisrequiresthesessionmiddleware)
req.session.uid=user._key;
req.session.twitterToken=authData.twitterToken;
req.session.twitterSecret=authData.twitterSecret;
req.sessionStorage.save(req.session);
//Redirecttothedefaultroute
res.redirect(303,req.makeAbsolute('/'));
},'oauth1_callback')
.queryParam('oauth_token',joi.string().optional())
.queryParam('oauth_verifier',joi.string().optional());
CreatinganOAuth1.0aclient
createOAuth1Client(options):OAuth1Client
CreatesanOAuth1.0aclient.
Arguments
options: Object
Anobjectwiththefollowingproperties:
requestTokenEndpoint: string
Thefully-qualifiedURLoftheprovider'sTemporaryCredentialsRequestendpoint.ThisURLisusedtofetchthe
unauthenticatedtemporarycredentialsthatwillbeusedtogeneratetheauthorizationredirectfortheuser.
authEndpoint: string
Thefully-qualifiedURLoftheprovider'sResourceOwnerAuthorizationendpoint.ThisistheURLtheuserwillberedirected
toinordertoauthorizetheOAuthconsumer(i.e.yourservice).
accessTokenEndpoint: string
Thefully-qualifiedURLoftheprovider'sTokenRequestendpoint.ThisURLisusedtoexchangetheauthenticatedtemporary
credentialsreceivedfromtheauthorizationredirectfortheactualtokencredentialsthatcanbeusedtomakerequeststotheAPI
server.
activeUserEndpoint: string(optional)
Thefully-qualifiedURLoftheprovider'sendpointforfetchingdetailsaboutthecurrentuser.
clientId: string
Theapplication'sClientID(orConsumerKey)fortheprovider.
clientS ecret: string
Theapplication'sClientSecret(orConsumerSecret)fortheprovider.
signatureMethod: string(Default: "HMAC-SHA1")
ThecryptographicmethodthatwillbeusedtosignOAuth1.0arequests.Only "HMAC-SHA1-"and "PLAINTEXT"are
supportedatthistime.
Notethatmanyprovidersmaynotimplement "PLAINTEXT"asitexposestheClientSecretand oauth_token_secretinsteadof
generatingasignature.
ReturnsanOAuth1.0aclientforthegivenprovider.
SettingupOAuth1.0aforTwitter
IfyouwanttouseTwitterastheOAuth1.0aprovider,usethefollowingoptions:
274
OAuth1.0a
requestTokenEndpoint: https://api.twitter.com/oauth/request_token
authEndpoint: https://api.twitter.com/oauth/authorize
accessTokenEndpoint: https://api.twitter.com/oauth/access_token
activeUserEndpoint: https://api.twitter.com/1.1/account/verify_credentials.json
YoualsoneedtoobtainaclientIDandclientsecretfromTwitter:
1. CreatearegularaccountatTwitteroruseanexistingaccountyouown.
2. VisittheTwitterApplicationM anagementdashboardandsigninwithyourTwitteraccount.
3. ClickonCreateNewAppandfollowtheinstructionsprovided.TheCallbackURLshouldmatchyouroauth_callbacklater.You
maybepromptedtoaddamobilephonenumbertoyouraccountandverifyit.
4. OpentheKeysandAccessTonestab,thennotedowntheConsumerKeyandConsumerSecret.
5. SettheoptionclientIdtotheConsumerKeyandtheoptionclientSecrettotheConsumerSecret.
Notethatifyouonlyneedread-onlyaccesstopublicinformation,youcanalsousetheclientIdandclientSecretdirectlywithoutOAuth
1.0a.
SeeTwitterRESTAPIReferenceDocumentation.
SettingupOAuth1.0aforXING
IfyouwanttouseXINGastheOAuth1.0aprovider,usethefollowingoptions:
requestTokenEndpoint: https://api.xing.com/v1/request_token
authEndpoint: https://api.xing.com/v1/authorize
accessTokenEndpoint: https://api.xing.com/v1/access_token
activeUserEndpoint: https://api.xing.com/v1/users/me
YoualsoneedtoobtainaclientIDandclientsecretfromXING:
1. CreatearegularaccountatXINGoruseanexistingaccountyouown.
2. VisittheXINGDeveloperpageandsigninwithyourXINGaccount.
3. ClickonCreateappandnotedowntheConsumerkeyandConsumersecret.
4. SettheoptionclientIdtotheConsumerkeyandtheoptionclientSecrettotheConsumersecret.
SeeXINGDeveloperDocumentation.
SettingupOAuth1.0aforTumblr
IfyouwanttouseTumblrastheOAuth1.0aprovider,usethefollowingoptions:
requestTokenEndpoint: https://www.tumblr.com/oauth/request_token
authEndpoint: https://www.tumblr.com/oauth/authorize
accessTokenEndpoint: https://www.tumblr.com/oauth/access_token
activeUserEndpoint: https://api.tumblr.com/v2/user/info
YoualsoneedtoobtainaclientIDandclientsecretfromTumblr:
1. CreatearegularaccountatTumblroruseanexistingaccountyouown.
2. VisittheTumblrApplicationsdashboard.
3. ClickonRegisterapplication,thenfollowtheinstructionsprovided.TheDefaultcallbackURLshouldmatchyouroauth_callback
later.
4. NotedowntheOAuthConsumerKeyandSecretKey.Thesecretmaybehiddenbydefault.
5. SettheoptionclientIdtotheOAuthConsumerKeyandtheoptionclientSecrettotheSecretKey.
SeeTumblrAPIDocumentation.
Fetchanunauthenticatedrequesttoken
oauth1.fetchRequestToken(oauth_callback,opts)
Fetchesan oauth_tokenthatcanbeusedtocreateanauthorizationURLthatredirectstothegiven oauth_callbackonconfirmation.
275
OAuth1.0a
PerformsaPOSTresponsetotherequestTokenEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
Arguments
oauth_callback: string
Thefully-qualifiedURLofyourapplication'sOAuth1.0acallback.
opts: Object(optional)
Anobjectwithadditionalqueryparameterstoincludeintherequest.
SeeRFC5849.
Returnstheparsedresponseobject.
GettheauthorizationURL
oauth1.getAuthUrl(oauth_token,opts):string
GeneratestheauthorizationURLfortheauthorizationendpoint.
Arguments
oauth_token: string
The oauth_tokenpreviouslyreturnedby fetchRequestToken.
opts:(optional)
AnobjectwithadditionalqueryparameterstoaddtotheURL.
SeeRFC5849.
Returnsafully-qualifiedURLfortheauthorizationendpointoftheproviderbyappendingthe oauth_tokenandanyadditional
argumentsfromoptstotheauthEndpoint.
Examples
constrequestToken=oauth1.fetchRequestToken(oauth_callback);
if(requestToken.oauth_callback_confirmed!=='true'){
thrownewError('ProvidercouldnotconfirmOAuth1.0callback');
}
constauthUrl=oauth1.getAuthUrl(requestToken.oauth_token);
Exchangeanauthenticatedrequesttokenforanaccesstoken
oauth1.exchangeRequestToken(oauth_token,oauth_verifier,opts)
TakesapairofauthenticatedtemporarycredentialspassedtothecallbackURLbytheproviderandexchangesitforan oauth_tokenand
oauth_token_secretthancanbeusedtoperformauthenticatedrequeststotheOAuth1.0aprovider.
PerformsaPOSTresponsetotheaccessTokenEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
Arguments
oauth_token: string
The oauth_tokenpassedtothecallbackURLbytheprovider.
oauth_verifier: string
The oauth_verifierpassedtothecallbackURLbytheprovider.
opts: Object(optional)
276
OAuth1.0a
Anobjectwithadditionalqueryparameterstoincludeintherequest.
SeeRFC5849.
Returnstheparsedresponseobject.
Fetchtheactiveuser
oauth1.fetchActiveUser(oauth_token,oauth_token_secret,opts):Object
Fetchesdetailsoftheactiveuser.
PerformsaGETresponsetotheactiveUserEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
Returns nulliftheactiveUserEndpointisnotconfigured.
Arguments
oauth_token: string
AnOAuth1.0aaccesstokenasreturnedbyexchangeRequestToken.
oauth_token_secret: string
AnOAuth1.0aaccesstokensecretasreturnedbyexchangeRequestToken.
opts: Object(optional)
Anobjectwithadditionalqueryparameterstoincludeintherequest.
SeeRFC5849.
Returnstheparsedresponseobject.
Examples
constauthData=oauth1.exchangeRequestToken(oauth_token,oauth_verifier);
constuserData=oauth1.fetchActiveUser(authData.oauth_token,authData.oauth_token_secret);
Createanauthenticatedrequestobject
oauth1.createSignedRequest(method,url,parameters,oauth_token,oauth_token_secret)
CreatesarequestobjectthatcanbeusedtoperformarequesttotheOAuth1.0aproviderwiththeprovidedtokencredentials.
Arguments
method: string
HTTPmethodtherequestwilluse,e.g. "POST".
url: string
Thefully-qualifiedURLoftheprovidertherequestwillbeperformedagainst.
TheURLmayoptionallycontainanynumberofqueryparameters.
parameters: string|Object|null
Anadditionalobjectorquerystringcontainingqueryparametersorbodyparametersthatwillbepartofthesignedrequest.
oauth_token: string
AnOAuth1.0aaccesstokenasreturnedbyexchangeRequestToken.
oauth_token_secret: string
277
OAuth1.0a
AnOAuth1.0aaccesstokensecretasreturnedbyexchangeRequestToken.
Returnsanobjectwiththreeproperties:
url:ThenormalizedURLwithoutanyqueryparameters.
qs:Anormalizedquerystringcontainingallparametersandqueryparameters.
headers:Anobjectcontainingthefollowingproperties:
accept:Thestring "application/json".
authorization:AnOAuthauthorizationheadercontainingallOAuthparametersandtherequestsignature.
Examples
Fetchalistoftweetsmentioning @arangodb:
constrequest=require('@arangodb/request');
constreq=oauth1.createSignedRequest(
'GET',
'https://api.twitter.com/1.1/search/tweets.json',
{q:'@arangodb'},
authData.oauth_token,
authData.oauth_token_secret
);
constres=request(req);
console.log(res.json.statuses);
Signingamorecomplexrequest:
consturl='https://api.example.com/v1/timeline?visible=public';
constparams={hello:'world',longcat:'islong'};
constreq=oauth1.createSignedRequest(
'POST',
url,//URLincludesaqueryparameterthatwillbesigned
params,//Requestbodyneedstobesignedtoo
authData.oauth_token,
authData.oauth_token_secret
);
constres=request.post(url,{
form:params,
headers:{
accept:'application/x-www-form-urlencoded',
//Authorizationheaderincludesthesignature
authorization:req.headers.authorization
}
});
console.log(res.json);
278
OAuth2.0
OAuth2.0
constcreateOAuth2Client=require('@arangodb/foxx/oauth2');
TheOAuth2moduleprovidesabstractionsoverOAuth2.0providerslikeFacebook,GitHubandGoogle.
Examples
Thefollowingextendstheusermanagementexample:
constcrypto=require('@arangodb/crypto');
constrouter=createRouter();
constoauth2=createOAuth2Client({
//We'lluseFacebookforthisexample
authEndpoint:'https://www.facebook.com/dialog/oauth',
tokenEndpoint:'https://graph.facebook.com/oauth/access_token',
activeUserEndpoint:'https://graph.facebook.com/v2.0/me',
clientId:'keyboardcat',
clientSecret:'keyboardcat'
});
module.context.use('/oauth2',router);
//Seetheusermanagementexampleforsettingupthe
//sessionsandusersobjectsusedinthisexample
router.use(sessions);
router.post('/auth',function(req,res){
constcsrfToken=crypto.genRandomAlphaNumbers(32);
consturl=req.reverse('oauth2_callback',{csrfToken});
constredirect_uri=req.makeAbsolute(url);
//SetCSRFcookieforfiveminutes
res.cookie('oauth2_csrf_token',csrfToken,{ttl:60*5});
//Redirecttotheprovider'sauthorizationURL
res.redirect(303,oauth2.getAuthUrl(redirect_uri));
});
router.get('/auth',function(req,res){
//Someproviderspasserrorsasqueryparameter
if(req.queryParams.error){
res.throw(500,`Providererror:${req.queryParams.error}`)
}
//MakesureCSRFcookiematchestheURL
constexpectedToken=req.cookie('oauth2_csrf_token');
if(!expectedToken||req.queryParams.csrfToken!==expectedToken){
res.throw(400,'CSRFmismatch.');
}
//MakesuretheURLcontainsagranttoken
if(!req.queryParams.code){
res.throw(400,'Providerdidnotpassgranttoken.');
}
//Reconstructtheredirect_uriusedforthegranttoken
consturl=req.reverse('oauth2_callback');
constredirect_uri=req.makeAbsolute(url);
//Fetchanaccesstokenfromtheprovider
constauthData=oauth2.exchangeGrantToken(
req.queryParams.code,
redirect_uri
);
constfacebookToken=authData.access_token;
//Fetchtheactiveuser'sprofileinfo
constprofile=oauth2.fetchActiveUser(facebookToken);
constfacebookId=profile.id;
//TrytofindanexistinguserwiththeuserID
//(thisrequirestheuserscollection)
letuser=users.firstExample({facebookId});
if(user){
//UpdatethefacebookTokenifithaschanged
if(user.facebookToken!==facebookToken){
users.update(user,{facebookToken});
}
}else{
279
OAuth2.0
//Createanewuserdocument
user={
username:`fb:${facebookId}`,
facebookId,
facebookToken
}
constmeta=users.save(user);
Object.assign(user,meta);
}
//Logtheuserin(thisrequiresthesessionmiddleware)
req.session.uid=user._key;
req.session.facebookToken=authData.facebookToken;
req.sessionStorage.save(req.session);
//Redirecttothedefaultroute
res.redirect(303,req.makeAbsolute('/'));
},'oauth2_callback')
.queryParam('error',joi.string().optional())
.queryParam('csrfToken',joi.string().optional())
.queryParam('code',joi.string().optional());
CreatinganOAuth2.0client
createOAuth2Client(options):OAuth2Client
CreatesanOAuth2.0client.
Arguments
options: Object
Anobjectwiththefollowingproperties:
authEndpoint: string
Thefully-qualifiedURLoftheprovider'sauthorizationendpoint.
tokenEndpoint: string
Thefully-qualifiedURLoftheprovider'stokenendpoint.
refreshEndpoint: string(optional)
Thefully-qualifiedURLoftheprovider'srefreshtokenendpoint.
activeUserEndpoint: string(optional)
Thefully-qualifiedURLoftheprovider'sendpointforfetchingdetailsaboutthecurrentuser.
clientId: string
Theapplication'sClientID(orAppID)fortheprovider.
clientS ecret: string
Theapplication'sClientSecret(orAppSecret)fortheprovider.
ReturnsanOAuth2.0clientforthegivenprovider.
SettingupOAuth2.0forFacebook
IfyouwanttouseFacebookastheOAuth2.0provider,usethefollowingoptions:
authEndpoint: https://www.facebook.com/dialog/oauth
tokenEndpoint: https://graph.facebook.com/oauth/access_token
activeUserEndpoint: https://graph.facebook.com/v2.0/me
YoualsoneedtoobtainaclientIDandclientsecretfromFacebook:
1. CreatearegularaccountatFacebookoruseanexistingaccountyouown.
2. VisittheFacebookDeveloperspage.
280
OAuth2.0
3. ClickonAppsinthemenu,thenselectRegisterasaDeveloper(theonlyoption)andfollowtheinstructionsprovided.Youmay
needtoverifyyouraccountbyphone.
4. ClickonAppsinthemenu,thenselectCreateaNewAppandfollowtheinstructionsprovided.
5. Opentheappdashboard,thennotedowntheAppIDandAppSecret.Thesecretmaybehiddenbydefault.
6. ClickonSettings,thenAdvancedandenteroneormoreValidOAuthredirectURIs.Atleastoneofthemmustmatchyour
redirect_urilater.Don'tforgettosaveyourchanges.
7. SettheoptionclientIdtotheAppIDandtheoptionclientSecrettotheAppSecret.
SettingupOAuth2.0forGitHub
IfyouwanttouseGitHubastheOAuth2.0provider,usethefollowingoptions:
authEndpoint: https://github.com/login/oauth/authorize?scope=user
tokenEndpoint: https://github.com/login/oauth/access_token
activeUserEndpoint: https://api.github.com/user
YoualsoneedtoobtainaclientIDandclientsecretfromGitHub:
1. CreatearegularaccountatGitHuboruseanexistingaccountyouown.
2. GotoAccountSettings>Applications>Registernewapplication.
3. ProvideanauthorizationcallbackURL.Thismustmatchyourredirect_urilater.
4. Fillintheotherrequireddetailsandfollowtheinstructionsprovided.
5. Opentheapplicationpage,thennotedowntheClientIDandClientSecret.
6. SettheoptionclientIdtotheClientIDandtheoptionclientSecrettotheClientSecret.
SettingupOAuth2.0forGoogle
IfyouwanttouseGoogleastheOAuth2.0provider,usethefollowingoptions:
authEndpoint: https://accounts.google.com/o/oauth2/auth?access_type=offline&scope=profile
tokenEndpoint: https://accounts.google.com/o/oauth2/token
activeUserEndpoint: https://www.googleapis.com/plus/v1/people/me
YoualsoneedtoobtainaclientIDandclientsecretfromGoogle:
1. CreatearegularaccountatGoogleoruseanexistingaccountyouown.
2. VisittheGoogleDevelopersConsole.
3. ClickonCreateProject,thenfollowtheinstructionsprovided.
4. Whenyourprojectisready,opentheprojectdashboard,thenclickonEnableanAPI.
5. EnabletheGoogle+APItoallowyourapptodistinguishbetweendifferentusers.
6. OpentheCredentialspageandclickCreatenewClientID,thenfollowtheinstructionsprovided.AtleastoneAuthorizedRedirect
URImustmatchyourredirect_urilater.AtleastoneAuthorizedJavaScriptOriginmustmatchyourapp'sfully-qualifieddomain.
7. WhentheClientIDisready,notedowntheClientIDandClientsecret.
8. SettheoptionclientIdtotheClientIDandtheoptionclientSecrettotheClientsecret.
GettheauthorizationURL
oauth2.getAuthUrl(redirect_uri,args):string
GeneratestheauthorizationURLfortheauthorizationendpoint.
Arguments
redirect_uri: string
Thefully-qualifiedURLofyourapplication'sOAuth2.0callback.
args:(optional)
Anobjectwithanyofthefollowingproperties:
response_type: string(Default: "code")
281
OAuth2.0
SeeRFC6749.
Returnsafully-qualifiedURLfortheauthorizationendpointoftheproviderbyappendingtheclientIDandanyadditionalarguments
fromargstotheauthEndpoint.
Exchangeagrantcodeforanaccesstoken
oauth2.exchangeGrantToken(code,redirect_uri)
Exchangesagrantcodeforanaccesstoken.
PerformsaPOSTresponsetothetokenEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
Arguments
code: string
Agrantcodereturnedbytheprovider'sauthorizationendpoint.
redirect_uri: string
TheoriginalcallbackURLwithwhichthecodewasrequested.
args: Object(optional)
Anobjectwithanyofthefollowingproperties:
grant_type: string(Default: "authorization_code")
SeeRFC6749.
Returnstheparsedresponseobject.
Fetchtheactiveuser
oauth2.fetchActiveUser(access_token):Object
Fetchesdetailsoftheactiveuser.
PerformsaGETresponsetotheactiveUserEndpoint.
Throwsanexceptioniftheremoteserverrespondswithanemptyresponsebody.
Returns nulliftheactiveUserEndpointisnotconfigured.
Arguments
access_token: string
AnOAuth2.0accesstokenasreturnedbyexchangeGrantToken.
Returnstheparsedresponseobject.
Examples
constauthData=oauth2.exchangeGrantToken(code,redirect_uri);
constuserData=oauth2.fetchActiveUser(authData.access_token);
282
Transactions
Transactions
Startingwithversion1.3,ArangoDBprovidessupportforuser-definabletransactions.
TransactionsinArangoDBareatomic,consistent,isolated,anddurable(ACID).
TheseACIDpropertiesprovidethefollowingguarantees:
Theatomicityprinciplemakestransactionseithercompleteintheirentiretyorhavenoeffectatall.
Theconsistencyprincipleensuresthatnoconstraintsorotherinvariantswillbeviolatedduringorafteranytransaction.
Theisolationpropertywillhidethemodificationsofatransactionfromothertransactionsuntilthetransactioncommits.
Finally,thedurabilitypropositionmakessurethatoperationsfromtransactionsthathavecommittedwillbemadepersistent.The
amountoftransactiondurabilityisconfigurableinArangoDB,asisthedurabilityoncollectionlevel.
283
Transactioninvocation
Transactioninvocation
ArangoDBtransactionsaredifferentfromtransactionsinSQL.
InSQL,transactionsarestartedwithexplicitBEGINorSTARTTRANSACTIONcommand.Followinganyseriesofdataretrievalor
modificationoperations,anSQLtransactionisfinishedwithaCOMMITcommand,orrolledbackwithaROLLBACKcommand.There
maybeclient/servercommunicationbetweenthestartandthecommit/rollbackofanSQLtransaction.
InArangoDB,atransactionisalwaysaserver-sideoperation,andisexecutedontheserverinonego,withoutanyclientinteraction.All
operationstobeexecutedinsideatransactionneedtobeknownbytheserverwhenthetransactionisstarted.
TherearenoindividualBEGIN,COMMITorROLLBACKtransactioncommandsinArangoDB.Instead,atransactioninArangoDBis
startedbyprovidingadescriptionofthetransactiontothedb._executeTransactionJavaScriptfunction:
db._executeTransaction(description);
Thisfunctionwillthenautomaticallystartatransaction,executeallrequireddataretrievaland/ormodificationoperations,andattheend
automaticallycommitthetransaction.Ifanerroroccursduringtransactionexecution,thetransactionisautomaticallyaborted,andall
changesarerolledback.
Executetransaction
executesatransaction db._executeTransaction(object)
Executesaserver-sidetransaction,asspecifiedbyobject.
objectmusthavethefollowingattributes:
collections:asub-objectthatdefineswhichcollectionswillbeusedinthetransaction.collectionscanhavetheseattributes:
read:asinglecollectionoralistofcollectionsthatwillbeusedinthetransactioninread-onlymode
write:asinglecollectionoralistofcollectionsthatwillbeusedinthetransactioninwriteorreadmode.
action:aJavascriptfunctionorastringwithJavascriptcodecontainingalltheinstructionstobeexecutedinsidethetransaction.If
thecoderunsthroughsuccessfully,thetransactionwillbecommittedattheend.Ifthecodethrowsanexception,thetransactionwill
berolledbackandalldatabaseoperationswillberolledback.
Additionally,objectcanhavethefollowingoptionalattributes:
waitForSync:booleanflagindicatingwhetherthetransactionisforcedtobesynchronous.
lockTimeout:anumericvaluethatcanbeusedtosetatimeoutforwaitingoncollectionlocks.Ifnotspecified,adefaultvaluewillbe
used.SettinglockTimeoutto0willmakeArangoDBnottimeoutwaitingforalock.
params:optionalargumentspassedtothefunctionspecifiedinaction.
ThefollowingattributescanbeusedfortransactionsintheRocksDBstorageengine:
maxTransactionSize:transactionsizelimitinbytes
intermediateCommitSize:maximumtotalsizeofoperationsafterwhichanintermediatecommitisperformedautomatically
intermediateCommitCount:maximumnumberofoperationsafterwhichanintermediatecommitisperformedautomatically
Declarationofcollections
Allcollectionswhicharetoparticipateinatransactionneedtobedeclaredbeforehand.Thisisanecessitytoensureproperlockingand
isolation.
Collectionscanbeusedinatransactioninwritemodeorinread-onlymode.
Ifanydatamodificationoperationsaretobeexecuted,thecollectionmustbedeclaredforuseinwritemode.Thewritemodeallows
modifyingandreadingdatafromthecollectionduringthetransaction(i.e.thewritemodeincludesthereadmode).
Contrary,usingacollectioninread-onlymodewillonlyallowperformingreadoperationsonacollection.Anyattempttowriteintoa
collectionusedinread-onlymodewillmakethetransactionfail.
284
Transactioninvocation
Collectionsforatransactionaredeclaredbyprovidingtheminthecollectionsattributeoftheobjectpassedtothe_executeTransaction
function.Thecollectionsattributehasthesub-attributesreadandwrite:
db._executeTransaction({
collections:{
write:["users","logins"],
read:["recommendations"]
}
});
readandwriteareoptionalattributes,andonlyneedtobespecifiediftheoperationsinsidethetransactionsdemandforit.
Thecontentsofreadorwritecaneachbelistsarrayscollectionnamesorasinglecollectionname(asastring):
db._executeTransaction({
collections:{
write:"users",
read:"recommendations"
}
});
Note:Itiscurrentlyoptionaltospecifycollectionsforread-onlyaccess.Evenwithoutspecifyingthem,itisstillpossibletoreadfrom
suchcollectionsfromwithinatransaction,butwithrelaxedisolation.PleaserefertoTransactionsLockingformoredetails.
Inordertomakeatransactionfailwhenanon-declaredcollectionisusedinsideforreading,theoptionalallowImplicitsub-attributeof
collectionscanbesettofalse:
db._executeTransaction({
collections:{
read:"recommendations",
allowImplicit:false/*thisdisallowsreadaccesstoothercollections
thanspecified*/
},
action:function(){
vardb=require("@arangodb").db;
returndb.foobar.toArray();/*willfailbecausedb.foobarmustnotbeaccessed
forreadinginsidethistransaction*/
}
});
ThedefaultvalueforallowImplicitistrue.Write-accessingcollectionsthathavenotbeendeclaredinthecollectionsarrayisnever
possible,regardlessofthevalueofallowImplicit.
Declarationofdatamodificationandretrievaloperations
AlldatamodificationandretrievaloperationsthataretobeexecutedinsidethetransactionneedtobespecifiedinaJavascriptfunction,
usingtheactionattribute:
db._executeTransaction({
collections:{
write:"users"
},
action:function(){
//alloperationsgohere
}
});
AnyvalidJavascriptcodeisallowedinsideactionbutthecodemayonlyaccessthecollectionsdeclaredincollections.actionmaybea
Javascriptfunctionasshownabove,orastringrepresentationofaJavascriptfunction:
db._executeTransaction({
collections:{
write:"users"
},
action:"function(){doSomething();}"
});
285
Transactioninvocation
Pleasenotethatanyoperationsspecifiedinactionwillbeexecutedontheserver,inaseparatescope.Variableswillbeboundlate.
AccessinganyJavaScriptvariablesdefinedontheclient-sideorinsomeotherservercontextfrominsideatransactionmaynotwork.
Instead,anyvariablesusedinsideactionshouldbedefinedinsideactionitself:
db._executeTransaction({
collections:{
write:"users"
},
action:function(){
vardb=require(...).db;
db.users.save({...});
}
});
Whenthecodeinsidetheactionattributeisexecuted,thetransactionisalreadystartedandallrequiredlockshavebeenacquired.When
thecodeinsidetheactionattributefinishes,thetransactionwillautomaticallycommit.Thereisnoexplicitcommitcommand.
Tomakeatransactionabortandrollbackallchanges,anexceptionneedstobethrownandnotcaughtinsidethetransaction:
db._executeTransaction({
collections:{
write:"users"
},
action:function(){
vardb=require("@arangodb").db;
db.users.save({_key:"hello"});
//willabortandrollbackthetransaction
throw"doh!";
}
});
Thereisnoexplicitabortorrollbackcommand.
Asmentionedearlier,atransactionwillcommitautomaticallywhentheendoftheactionfunctionisreachedandnoexceptionhasbeen
thrown.Inthiscase,theusercanreturnanylegalJavaScriptvaluefromthefunction:
db._executeTransaction({
collections:{
write:"users"
},
action:function(){
vardb=require("@arangodb").db;
db.users.save({_key:"hello"});
//willcommitthetransactionandreturnthevalue"hello"
return"hello";
}
});
Customexceptions
Onemaywishtodefinecustomexceptionsinsideofatransaction.Tohavetheexceptionpropagateupwardsproperly,pleasethrowan
aninstanceofbaseJavaScript Errorclassoraderivative.Tospecifyanerrornumber,includeitasthe errorNumberfield.Asan
example:
db._executeTransaction({
collections:{},
action:function(){
varerr=newError('Myerrorcontext');
err.errorNumber=1234;
throwerr;
}
});
286
Transactioninvocation
Note:Inpreviousversions,customexceptionswhichdidnothavean Error-likeformweresimplyconvertedtostringsandexposedin
the exceptionfieldofthereturnederror.Thisisnolongerthecase,asithadthepotentialtoleakunwantedinformationifimproperly
used.
Examples
Thefirstexamplewillwrite3documentsintoacollectionnamedc1.Thec1collectionneedstobedeclaredinthewriteattributeofthe
collectionsattributepassedtotheexecuteTransactionfunction.
Theactionattributecontainstheactualtransactioncodetobeexecuted.Thiscodecontainsalldatamodificationoperations(3inthis
example).
//setup
db._create("c1");
db._executeTransaction({
collections:{
write:["c1"]
},
action:function(){
vardb=require("@arangodb").db;
db.c1.save({_key:"key1"});
db.c1.save({_key:"key2"});
db.c1.save({_key:"key3"});
}
});
db.c1.count();//3
Abortingthetransactionbythrowinganexceptionintheactionfunctionwillrevertallchanges,soasifthetransactionneverhappened:
//setup
db._create("c1");
db._executeTransaction({
collections:{
write:["c1"]
},
action:function(){
vardb=require("@arangodb").db;
db.c1.save({_key:"key1"});
db.c1.count();//1
db.c1.save({_key:"key2"});
db.c1.count();//2
throw"doh!";
}
});
db.c1.count();//0
Theautomaticrollbackisalsoexecutedwhenaninternalexceptionisthrownatsomepointduringtransactionexecution:
//setup
db._create("c1");
db._executeTransaction({
collections:{
write:["c1"]
},
action:function(){
vardb=require("@arangodb").db;
db.c1.save({_key:"key1"});
//willthrowduplicateakeyerror,notexplicitlyrequestedbytheuser
db.c1.save({_key:"key1"});
//we'llnevergethere...
}
});
db.c1.count();//0
287
Transactioninvocation
Asrequiredbytheconsistencyprinciple,abortingorrollingbackatransactionwillalsorestoresecondaryindexestothestateat
transactionstart.
Cross-collectiontransactions
There'salsothepossibilitytorunatransactionacrossmultiplecollections.Inthiscase,multiplecollectionsneedtobedeclaredinthe
collectionsattribute,e.g.:
//setup
db._create("c1");
db._create("c2");
db._executeTransaction({
collections:{
write:["c1","c2"]
},
action:function(){
vardb=require("@arangodb").db;
db.c1.save({_key:"key1"});
db.c2.save({_key:"key2"});
}
});
db.c1.count();//1
db.c2.count();//1
Again,throwinganexceptionfrominsidetheactionfunctionwillmakethetransactionabortandrollbackallchangesinallcollections:
//setup
db._create("c1");
db._create("c2");
db._executeTransaction({
collections:{
write:["c1","c2"]
},
action:function(){
vardb=require("@arangodb").db;
for(vari=0;i<100;++i){
db.c1.save({_key:"key"+i});
db.c2.save({_key:"key"+i});
}
db.c1.count();//100
db.c2.count();//100
//abort
throw"doh!"
}
});
db.c1.count();//0
db.c2.count();//0
288
Passingparameters
Passingparameterstotransactions
Arbitraryparameterscanbepassedtotransactionsbysettingtheparamsattributewhendeclaringthetransaction.Thisfeatureishandy
tore-usethesametransactioncodeformultiplecallsbutwithdifferentparameters.
Abasicexample:
db._executeTransaction({
collections:{},
action:function(params){
returnparams[1];
},
params:[1,2,3]
});
Theaboveexamplewillreturn2.
Someexamplethatusescollections:
db._executeTransaction({
collections:{
write:"users",
read:["c1","c2"]
},
action:function(params){
vardb=require('@arangodb').db;
vardoc=db.c1.document(params['c1Key']);
db.users.save(doc);
doc=db.c2.document(params['c2Key']);
db.users.save(doc);
},
params:{
c1Key:"foo",
c2Key:"bar"
}
});
289
Lockingandisolation
LockingandIsolation
Transactionsneedtospecifyfromwhichcollectionstheywillreaddataandwhichcollectionstheyintenddomodify.Thiscanbedone
bysettingtheread,write,orexclusiveattributesinthecollectionsattributeofthetransaction:
db._executeTransaction({
collections:{
read:"users",
write:["test","log"]
},
action:function(){
constdb=require("@arangodb").db;
db.users.toArray().forEach(function(doc){
db.log.insert({value:"removeduser:"+doc.name});
db.test.remove(doc._key);
});
}
});
writeheremeanswriteaccesstothecollection,andalsoincludesanyreadaccesses.exclusiveisasynonymforwriteintheM M Files
engine,becausebothexclusiveandwritewillacquirecollection-levellocksinthisengine.IntheRocksDBengine,exclusivemeans
exclusivewriteaccesstothecollection,andwritemeans(shared)writeaccesstothecollection,whichcanbeinterleavedwithwrite
accessesbyotherconcurrenttransactions.
MMFilesengine
TheMMFilesengineusesthefollowinglockingmechanismstoserializetransactionsonthesamedata:
Allcollectionsspecifiedinthecollectionsattributearelockedintherequestedmode(readorwrite)attransactionstart.Lockingof
multiplecollectionsisperformedinalphabeticalorder.Whenatransactioncommitsorrollsback,alllocksarereleasedinreverseorder.
Thelockingorderisdeterministictoavoiddeadlocks.
Whilelocksareheld,modificationsbyothertransactionstothecollectionsparticipatinginthetransactionareprevented.Atransaction
willthusseeaconsistentviewoftheparticipatingcollections'data.
Additionally,atransactionwillnotbeinterruptedorinterleavedwithanyotherongoingoperationsonthesamecollection.Thismeans
eachtransactionwillruninisolation.Atransactionshouldneverseeuncommittedorrolledbackmodificationsbyothertransactions.
Additionally,readsinsideatransactionarerepeatable.
Notethattheaboveistrueonlyforallcollectionsthataredeclaredinthecollectionsattributeofthetransaction.
RocksDBengine
TheRocksDBenginedoesnotlockanycollectionsparticipatinginatransactionforread.Readoperationscanruninparalleltootherread
orwriteoperationsonthesamecollections.
Forallcollectionsthatareusedinwritemode,theRocksDBenginewillinternallyacquirea(shared)readlock.Thismeansthatmany
writerscanmodifydatainthesamecollectioninparallel(andalsoruninparalleltoongoingreads).However,iftwoconcurrent
transactionsattempttomodifythesamedocumentorindexentry,therewillbeawrite-writeconflict,andoneofthetransactionswill
abortwitherror1200("conflict").Itisthenuptoclientapplicationstoretrythefailedtransactionoracceptthefailure.
Inordertoguardlong-runningorcomplextransactionsagainstconcurrentoperationsonthesamedata,theRocksDBengineallowsto
accesscollectionsinexclusivemode.Exclusiveaccesseswillinternallyacquireawrite-lockonthecollections,sotheyarenotexecutedin
parallelwithanyotherwriteoperations.Readoperationscanstillbecarriedoutbyotherconcurrenttransactions.
Lazilyaddingcollections
Theremightbesituationswhendeclaringallcollectionsaprioriisnotpossible,forexample,becausefurthercollectionsaredeterminedby
adynamicAQLqueryinsidethetransaction,forexampleaqueryusingAQLgraphtraversal.
290
Lockingandisolation
Inthiscase,itwouldbeimpossibletoknowbeforehandwhichcollectiontolock,andthusitislegaltonotdeclarecollectionsthatwillbe
accessedinthetransactioninread-onlymode.Accessinganon-declaredcollectioninread-onlymodeduringatransactionwilladdthe
collectiontothetransactionlazily,andfetchdatafromthecollectionasusual.However,asthecollectionisaddedlazily,thereisno
isolationfromotherconcurrentoperationsortransactions.Readsfromsuchcollectionsarepotentiallynon-repeatable.
Examples:
db._executeTransaction({
collections:{
read:"users"
},
action:function(){
constdb=require("@arangodb").db;
/*ExecuteanAQLquerythattraversesagraphstartingata"users"vertex.
Itisyetunknownintowhichothercollectionsthequerymighttraverse*/
db._createStatement({
query:`FORvINANY"users/1234"connectionsRETURNv`
}).execute().toArray().forEach(function(d){
/*...*/
});
}
});
Thisautomaticlazyadditionofcollectionstoatransactionalsointroducesthepossibilityofdeadlocks.Deadlocksmayoccurifthereare
concurrenttransactionsthattrytoacquirelocksonthesamecollectionslazily.
Inordertomakeatransactionfailwhenanon-declaredcollectionisusedinsideatransactionforreading,theoptionalallowImplicitsubattributeofcollectionscanbesettofalse:
db._executeTransaction({
collections:{
read:"users",
allowImplicit:false
},
action:function(){
/*Thebelowquerywillnowfailbecausethecollection"connections"hasnot
beenspecifiedinthelistofcollectionsusedbythetransaction*/
constdb=require("@arangodb").db;
db._createStatement({
query:`FORvINANY"users/1234"connectionsRETURNv`
}).execute().toArray().forEach(function(d){
/*...*/
});
}
});
ThedefaultvalueforallowImplicitistrue.Write-accessingcollectionsthathavenotbeendeclaredinthecollectionsarrayisnever
possible,regardlessofthevalueofallowImplicit.
Ifusers/1234hasanedgeinconnections,linkingittoanotherdocumentintheuserscollection,thenthefollowingexplicitdeclarationwill
work:
db._executeTransaction({
collections:{
read:["users","connections"],
allowImplicit:false
},
/*...*/
Iftheedgepointstoadocumentinanothercollectionhowever,thenthequerywillfail,unlessthatothercollectionisaddedtothe
declarationaswell.
Notethatifadocumenthandleisusedasstartingpointforatraversal,e.g. FORvINANY"users/not_linked"...or FORvINANY{_id:
"users/not_linked"}...,thennoerrorisraisedinthecaseofthestartvertexnothavinganyedgestofollow,with allowImplicit:
falseandusersnotbeingdeclaredforreadaccess.AQLonlyseesastringanddoesnotconsideritareadaccess,unlessthereareedges
connectedtoit. FORvINANYDOCUMENT("users/not_linked")...willfailevenwithoutedges,asitisalwaysconsideredtobearead
accesstotheuserscollection.
291
Lockingandisolation
DeadlocksandDeadlockdetection
Adeadlockisasituationinwhichtwoormoreconcurrentoperations(usertransactionsorAQLqueries)trytoaccessthesameresources
(collections,documents)andneedtowaitfortheotherstofinish,butnoneofthemcanmakeanyprogress.
AgoodexampleforadeadlockistwoconcurrentlyexecutingtransactionsT1andT2thattrytoaccessthesamecollectionsbutthatneed
towaitforeachother.Inthisexample,transactionT1willwritetocollection c1,butwillalsoreaddocumentsfromcollection c2
withoutannouncingit:
db._executeTransaction({
collections:{
write:"c1"
},
action:function(){
constdb=require("@arangodb").db;
/*writeintoc1(announced)*/
db.c1.insert({foo:"bar"});
/*someoperationherethattakeslongtoexecute...*/
/*readfromc2(unannounced)*/
db.c2.toArray();
}
});
TransactionT2announcestowriteintocollection c2,butwillalsoreaddocumentsfromcollection c1withoutannouncingit:
db._executeTransaction({
collections:{
write:"c2"
},
action:function(){
vardb=require("@arangodb").db;
/*writeintoc2(announced)*/
db.c2.insert({bar:"baz"});
/*someoperationherethattakeslongtoexecute...*/
/*readfromc1(unannounced)*/
db.c1.toArray();
}
});
Intheaboveexample,adeadlockwilloccuriftransactionT1andT2havebothacquiredtheirwritelocks(T1forcollection c1andT2
forcollection c2)andarethentryingtoreadfromtheotherother(T1willreadfrom c2,T2willreadfrom c1).T1willthentryto
acquirethereadlockoncollection c2,whichispreventedbytransactionT2.T2howeverwillwaitforthereadlockoncollection c1,
whichispreventedbytransactionT1.
Incaseofsuchdeadlock,therewouldbenoprogressforanyoftheinvolvedtransactions,andnoneoftheinvolvedtransactionscould
evercomplete.Thisiscompletelyundesirable,sotheautomaticdeadlockdetectionmechanisminArangoDBwillautomaticallyabortone
ofthetransactionsinvolvedinsuchdeadlock.Abortingmeansthatallchangesdonebythetransactionwillberolledbackanderror29
( deadlockdetected)willbethrown.
Clientcode(AQLqueries,usertransactions)thataccessesmorethanonecollectionshouldbeawareofthepotentialofdeadlocksand
shouldhandletheerror29( deadlockdetected)properly,eitherbypassingtheexceptiontothecallerorretryingtheoperation.
Toavoidbothdeadlocksandnon-repeatablereads,allcollectionsusedinatransactionshouldbespecifiedinthe collectionsattribute
whenknowninadvance.Incasethisisnotpossiblebecausecollectionsareaddeddynamicallyinsidethetransaction,deadlocksmay
occurandthedeadlockdetectionmaykickinandabortthetransaction.
TheRocksDBengineusesdocument-levellocksandthereforewillnothaveadeadlockproblemoncollectionlevel.Iftwoconcurrent
transactionshowevermodifythesamedocumentsorindexentries,theRocksDBenginewillsignalawrite-writeconflictandabortoneof
thetransactionswitherror1200("conflict")automatically.
292
Lockingandisolation
293
Durability
Durability
Transactionsareexecutedinmainmemoryfirstuntilthereiseitherarollbackoracommit.Onrollback,nodatawillbewrittentodisk,
buttheoperationsfromthetransactionwillbereversedinmemory.
Oncommit,allmodificationsdoneinthetransactionwillbewrittentothecollectiondatafiles.Thesewriteswillbesynchronizedtodisk
ifanyofthemodifiedcollectionshasthewaitForSyncpropertysettotrue,orifanyindividualoperationinthetransactionwasexecuted
withthewaitForSyncattribute.Additionally,transactionsthatmodifydatainmorethanonecollectionareautomaticallysynchronizedto
disk.Thissynchronizationisdonetonotonlyensuredurability,buttoalsoensureconsistencyincaseofaservercrash.
Thatmeansifyouonlymodifydatainasinglecollection,andthatcollectionhasitswaitForSyncpropertysettofalse,thewhole
transactionwillnotbesynchronizedtodiskinstantly,butwithasmalldelay.
Thereisthusthepotentialriskoflosingdatabetweenthecommitofthetransactionandtheactual(delayed)disksynchronization.This
isthesameaswritingintocollectionsthathavethewaitForSyncpropertysettofalseoutsideofatransaction.Incaseofacrashwith
waitForSyncsettofalse,theoperationsperformedinthetransactionwilleitherbevisiblecompletelyornotatall,dependingonwhether
thedelayedsynchronizationhadkickedinornot.
ToensuredurabilityoftransactionsonacollectionthathavethewaitForSyncpropertysettofalse,youcansetthewaitForSyncattribute
oftheobjectthatispassedtoexecuteTransaction.Thiswillforceasynchronizationofthetransactiontodiskevenforcollectionsthat
havewaitForSyncsettofalse:
db._executeTransaction({
collections:{
write:"users"
},
waitForSync:true,
action:function(){...}
});
Analternativeistoperformanoperationwithanexplicitsyncrequestinatransaction,e.g.
db.users.save({_key:"1234"},true);
Inthiscase,thetruevaluewillmakethewholetransactionbesynchronizedtodiskatthecommit.
Inanycase,ArangoDBwillgiveusersthechoiceofwhetherornottheywantfulldurabilityforsinglecollectiontransactions.Usingthe
delayedsynchronization(i.e.waitForSyncwithavalueoffalse)willpotentiallyincreasethroughputandperformanceoftransactions,but
willintroducetheriskoflosingthelastcommittedtransactionsinthecaseofacrash.
Incontrast,transactionsthatmodifydatainmorethanonecollectionareautomaticallysynchronizedtodisk.Thiscomesatthecostof
severaldisksync.Foramulti-collectiontransaction,thecalltothe_executeTransactionfunctionwillonlyreturnafterthedataofall
modifiedcollectionshasbeensynchronizedtodiskandthetransactionhasbeenmadefullydurable.Thisnotonlyreducestheriskof
losingdataincaseofacrashbutalsoensuresconsistencyafterarestart.
Incaseofaservercrash,anymulti-collectiontransactionsthatwerenotyetcommittedorinpreparationtobecommittedwillberolled
backonserverrestart.
Formulti-collectiontransactions,therewillbeatleastonedisksyncoperationpermodifiedcollection.M ulti-collectiontransactionsthus
haveapotentiallyhighercostthansinglecollectiontransactions.ThereisnoconfigurationtoturnoffdisksynchronizationformulticollectiontransactionsinArangoDB.Thedisksyncspeedofthesystemwillthusbethemostimportantfactorfortheperformanceof
multi-collectiontransactions.
294
Limitations
Limitations
InGeneral
TransactionsinArangoDBhavebeendesignedwithparticularusecasesinmind.Theywillbemainlyusefulforshortandsmalldata
retrievaland/ormodificationoperations.
Theimplementationisnotoptimizedforverylong-runningorveryvoluminousoperations,andmaynotbeusableforthesecases.
Onelimitationisthatatransactionoperationinformationmustfitintomainmemory.Thetransactioninformationconsistsofrecord
pointers,revisionnumbersandrollbackinformation.Theactualdatamodificationoperationsofatransactionarewrittentothewriteaheadloganddonotneedtofitentirelyintomainmemory.
Ongoingtransactionswillalsopreventthewrite-aheadlogsfrombeingfullygarbage-collected.Informationinthewrite-aheadlogfiles
cannotbewrittentocollectiondatafilesorbediscardedwhiletransactionsareongoing.
Toensureprogressofthewrite-aheadloggarbagecollection,transactionsshouldbekeptassmallaspossible,andbigtransactionsshould
besplitintomultiplesmallertransactions.
TransactionsinArangoDBcannotbenested,i.e.atransactionmustnotstartanothertransaction.Ifanattemptismadetocalla
transactionfrominsidearunningtransaction,theserverwillthrowerror1651(nestedtransactionsdetected).
ItisalsodisallowedtoexecuteusertransactiononsomeofArangoDB'sownsystemcollections.Thisshouldn'tbeaproblemforregular
usageassystemcollectionswillnotcontainuserdataandthereisnoneedtoaccessthemfromwithinausertransaction.
Someoperationsarenotallowedinsidetransactionsingeneral:
creationanddeletionofdatabases( db._createDatabase(), db._dropDatabase())
creationanddeletionofcollections( db._create(), db._drop(), db.<collection>.rename())
creationanddeletionofindexes( db.<collection>.ensureIndex(), db.<collection>.dropIndex())
Ifanattemptismadetocarryoutanyoftheseoperationsduringatransaction,ArangoDBwillabortthetransactionwitherrorcode1653
(disallowedoperationinsidetransaction).
Finally,allcollectionsthatmaybemodifiedduringatransactionmustbedeclaredbeforehand,i.e.usingthecollectionsattributeofthe
objectpassedtothe_executeTransactionfunction.Ifanyattemptismadetocarryoutadatamodificationoperationonacollectionthat
wasnotdeclaredinthecollectionsattribute,thetransactionwillbeabortedandArangoDBwillthrowerror1652unregisteredcollection
usedintransaction.Itislegaltonotdeclareread-onlycollections,butthisshouldbeavoidedifpossibletoreducetheprobabilityof
deadlocksandnon-repeatablereads.
PleaserefertoLockingandIsolationformoredetails.
InClusters
UsingasingleinstanceofArangoDB,multi-document/multi-collectionqueriesareguaranteedtobefullyACID.Thisismorethanmany
otherNoSQLdatabasesystemssupport.Inclustermode,single-documentoperationsarealsofullyACID.M ulti-document/multicollectionqueriesinaclusterarenotACID,whichisequallythecasewithcompetingdatabasesystems.Transactionsinaclusterwillbe
supportedinafutureversionofArangoDBandmaketheseoperationsfullyACIDaswell.Notethatfornon-shardedcollectionsina
cluster,thetransactionalpropertiesofasingleserverapply(fullyACID).
TransactionsintheRocksDBstorageengine
DataofongoingtransactionsisstoredinRAM .Transactionsthatgettoobig(intermsofnumberofoperationsinvolvedorthetotalsize
ofdatacreatedormodifiedbythetransaction)willbecommittedautomatically.Effectivelythismeansthatbigusertransactionsaresplit
intomultiplesmallerRocksDBtransactionsthatarecommittedindividually.TheentireusertransactionwillnotnecessarilyhaveACID
propertiesinthiscase.
ThefollowingglobaloptionscanbeusedtocontroltheRAM usageandautomaticintermediatecommitsfortheRocksDBengine:
295
Limitations
--rocksdb.max-transaction-size
Transactionsizelimit(inbytes).TransactionsstoreallkeysandvaluesinRAM ,solargetransactionsruntheriskofcausingout-ofmemorysitations.Thissettingallowsyoutoensurethatdoesnothappenbylimitingthesizeofanyindividualtransaction.Transactions
whoseoperationswouldconsumemoreRAM thanthisthresholdvaluewillabortautomaticallywitherror32("resourcelimitexceeded").
--rocksdb.intermediate-commit-size
Ifthesizeofalloperationsinatransactionreachesthisthreshold,thetransactioniscommittedautomaticallyandanewtransactionis
started.Thevalueisspecifiedinbytes.
--rocksdb.intermediate-commit-count
Ifthenumberofoperationsinatransactionreachesthisvalue,thetransactioniscommittedautomaticallyandanewtransactionis
started.
Theabovevaluescanalsobeadjustedpertransaction,bysettingthefollowingattributesinthecalltodb._executeTransaction():
maxTransactionSize:transactionsizelimitinbytes
intermediateCommitSize:maximumtotalsizeofoperationsafterwhichanintermediatecommitisperformedautomatically
intermediateCommitCount:maximumnumberofoperationsafterwhichanintermediatecommitisperformedautomatically
296
Deployment
Deployment
InthischapterwedescribevariouspossibilitiestodeployArangoDB.Inparticularfortheclustermode,therearedifferentwaysandwe
wanttohighlighttheiradvantagesanddisadvantages.Weevendocumentindetail,howtosetupaclusterbysimplystartingvarious
ArangoDBprocessesondifferentmachines,eitherdirectlyorusingDockercontainers.
Singleinstance
Cluster:DC/OS,ApacheM esosandM arathon
Cluster:Generic&Docker
AdvancedTopics
Cluster:Testsetuponalocalmachine
Cluster:Startingprocessesondifferentmachines
Cluster:LaunchinganArangoDBclusterusingDockercontainers
Agency
297
Singleinstance
Singleinstancedeployment
ThelatestofficialbuildsofArangoDBforallsupportedoperatingsystemsmaybeobtainedfromhttps://www.arangodb.com/download/.
Linuxremarks
Besidestheofficialimageswhichareprovidedforthemostpopularlinuxdistributionstherearealsoavarietyofunofficialimages
providedbythecommunity.Wearetrackingmostofthecommunitycontributions(includingneworupdatedimages)inournewsletter:
https://www.arangodb.com/category/newsletter/
Windowsremarks
PleasenotethatArangoDBwillonlyworkon64bit.
Docker
ThesimplestwaytodeployArangoDBisusingDocker.TogetageneralunderstandingofDockerhavealookattheirexcellent
documentation.
Authentication
TostarttheofficialDockercontaineryouwillhavetodecideonanauthenticationmethod.Otherwisethecontainerwon'tstart.
ProvideoneoftheargumentstoDockerasanenvironmentvariable.
Therearethreeoptions:
1. ARANGO_NO_AUTH=1
Disableauthenticationcompletely.Usefulforlocaltestingorforoperatinginatrustednetwork(withoutapublicinterface).
2. ARANGO_ROOT_PASSWORD=password
StartArangoDBwiththegivenpasswordforroot
3. ARANGO_RANDOM _ROOT_PASSWORD=1
LetArangoDBgeneratearandomrootpassword
Togetgoingquickly:
dockerrun-eARANGO_RANDOM_ROOT_PASSWORD=1arangodb/arangodb
ForanindepthguideaboutDockerandArangoDBpleasechecktheofficialdocumentation:https://hub.docker.com/r/arangodb/arangodb/.
Notethatweareusingtheimage arangodb/arangodbherewhichisalwaysthemostcurrentone.Thereisalsothe"official"onecalled
arangodbwhosedocumentationishere:https://hub.docker.com/_/arangodb/
298
Cluster:M esos,DC/OS
DistributeddeploymentusingApacheMesos
ArangoDBhasasophisticatedandyeteasywaytouseclustermode.Toleveragethefullclusterfeatureset(monitoring,scaling,
automaticfailoverandautomaticreplacementoffailednodes)youhavetorunArangoDBonsomekindofclustermanagementsystem.
CurrentlyArangoDBreliesonApacheM esosinthatmatter.M esosisaclusteroperatingsystemwhichpowerssomeoftheworlds
biggestdatacentersrunningseveralthousandsofnodes.
DC/OS
DC/OSistherecommendedwaytoinstallaclusterasiteasesmuchoftheprocesstoinstallaM esoscluster.Youcandeployitvery
quicklyonavarietyofcloudhostersorsetupyourownDC/OSlocally.DC/OSisasetoftoolsbuiltontopofApacheM esos.Apache
M esosisasocalled"DistributedClusterOperationSystem"andthecoreofDC/OS.ApacheM esoshastheconceptofsocalled
persistentvolumeswhichmakeitperfectlysuitableforadatabase.
Installing
FirstprepareaDC/OSclusterbygoingtohttps://dcos.ioandfollowingtheinstructionsthere.
DC/OScomeswithitsownpackagemanagement.Packagescanbeinstalledfromthesocalled"Universe".AsanofficialDC/OSpartner
ArangoDBcanbeinstalledfromtherestraightaway.
1. InstallingviaDC/OSUI
i. OpenyourbrowserandgototheDC/OSadmininterface
ii. Openthe"Universe"tab
iii. Locatearangodbandhit"InstallPackage"
iv. Press"InstallPackage"
2. InstallingviatheDC/OScommandline
i. Installthedcoscli
ii. Openaterminalandissue dcosinstallarangodb
Bothoptionsareessentiallydoingthesameinthebackground.BotharestartingArangoDBwithitsdefaultoptionsset.
Toreviewthedefaultoptionsusingthewebinterfacesimplyclick"AdvancedInstallation"inthewebinterface.Thereyouwillfindalist
ofoptionsincludingsomeexplanation.
ToreviewthedefaultoptionsusingtheCLIfirsttype dcospackagedescribe--configarangodb.Thiswillgiveyouaflatlistofdefault
settings.
Togetanexplanationofthevariouscommandlineoptionspleasecheckthelatestoptionshere(choosethemostrecentnumberandhave
alookat config.json):
https://github.com/mesosphere/universe/tree/version-3.x/repo/packages/A/arangodb
AfterinstallationDC/OSwillstartdeployingtheArangoDBclusterontheDC/OScluster.YoucanwatchArangoDBstartingonthe
"Services"tabinthewebinterface.OnceitislistedashealthyclickthelinknexttoitandyoushouldseetheArangoDBwebinterface.
ArangoDBMesosframework
AssoonasArangoDBwasdeployedM esoswillkeepyourclusterrunning.Thewebinterfacehasmanymonitoringfacilitiessobesure
tomakeyourselffamiliarwiththeDC/OSwebinterface.AsafaulttolerantsystemM esoswilltakecareofmostfailurescenarios
automatically.M esosdoesthatbyrunningArangoDBasasocalled"framework".Thisframeworkhasbeenspecificallybuilttokeep
ArangoDBrunninginahealthyconditionontheM esoscluster.Fromtimetotimeataskmightfail.TheArangoDBframeworkwillthen
takecareofreschedulingthefailedtask.Asitknowsabouttheveryspecificsofeachclustertaskanditsroleitwillautomaticallytake
careofmostfailurescenarios.
Toinspectwhattheframeworkisdoinggoto http://web-interface-url/mesosinyourbrowser.Locatethetask"arangodb"andinspect
stderrinthe"Sandbox".Thiscanbeofinterestforexamplewhenaslavegotlostandtheframeworkisreschedulingthetask.
299
Cluster:M esos,DC/OS
UsingArangoDB
TouseArangoDBasadatastoreinyourDC/OSclusteryoucanfacilitatetheservicediscoveryofDC/OS.Assumingyoudeployeda
standardArangoDBclusterthemesosdnswillknowabout arangodb.mesos.BydoingaSRVDNSrequest(checkthedocumentationof
mesosdns)youcanfindouttheportwheretheinternalHAProxyofArangoDBisrunning.Thiswillofferaroundrobinloadbalancerto
accessallArangoDBcoordinators.
ScalingArangoDB
TochangethesettingsofyourArangoDBClusteraccesstheArangoDBUIandhit"Nodes".Onthescaletabyouwillhavetheabilityto
scaleyourclusterupanddown.
AfterchangingthesettingstheArangoDBframeworkwilltakecareoftherest.Scalingyourclusterupisgenerallyastraightforward
operationasM esoswillsimplylaunchanothertaskandbedonewithit.Scalingdownisabitmorecomplicatedasthedatafirsthastobe
movedtosomeotherplacesothatwillnaturallytakesomewhatlonger.
Pleasenotethatscalingoperationsmightnotalwayswork.ForexampleiftheunderlyingM esosclusteriscompletelysaturatedwithits
runningtasksscalingupwillnotbepossible.Scalingdownmightalsofailduetotheclusternotbeingabletomoveallshardsofa
DBServertoanewdestinationbecauseofsizelimitations.BesuretochecktheoutputoftheArangoDBframework.
Deinstallation
DeinstallingArangoDBisabitmoredifficultasthereismuchstatebeingkeptintheM esosclusterwhichisnotautomaticallycleaned
up.Todeinstallfromthecommandlineusethefollowingoneliner:
dcosarangodbuninstall;dcospackageuninstallarangodb
Thiswillfirstcleanupthestateintheclusterandthenuninstallarangodb.
arangodb-cleanup-framework
Shouldyouforgettocleanupthestateyoucandosolaterbyusingthearangodb-cleanup-frameworkcontainer.Otherwiseyoumightnot
beabletodeployanewarangodbinstallation.
ThecleanupframeworkwillannounceitselfasanormalArangoDB.M esoswillrecognizethisandofferallpersistentvolumesitstillhas
forArangoDBtothisframework.Thecleanupframeworkwillthenproperlyfreethepersistentvolumes.Finallyitwillcleanupany
stateleftinzookeeper(thecentralconfigurationmanagerinaM esoscluster).
Todeploythecleanupframework,followtheinstructionsinthegithubrepository.Afterdeploymentwatchtheoutputinthesandboxof
theM esoswebinterface.Afterawhilethereshouldn'tbeanypersistentresourceoffersanymoreaseverythingwascleanedup.After
thatyoucandeletethecleanupframeworkagainviaM arathon.
ApacheMesosandMarathon
YoucanalsoinstallArangoDBonabareApacheM esosclusterprovidedthatM arathonisrunningonit.
Doingsohasthefollowingdownsides:
1. M anualM esosclustersetup
2. Youneedtoimplementyourownservicediscovery
3. Youaremissingthedcoscli
4. Installationanddeinstallationaretedious
5. YouneedtosetupsomekindofproxytunneltoaccessArangoDBfromtheoutside
6. Sparsemonitoringcapabilities
HoweverthesearethingswhichdonotinfluenceArangoDBitselfandoperatingyourclusterlikethisisfullysupported.
InstallingviaMarathon
ToinstallArangoDBviamarathonyouneedaproperconfigfile:
{
300
Cluster:M esos,DC/OS
"id":"arangodb",
"cpus":0.25,
"mem":256.0,
"ports":[0,0,0],
"instances":1,
"args":[
"framework",
"--framework_name=arangodb",
"--master=zk://172.17.0.2:2181/mesos",
"--zk=zk://172.17.0.2:2181/arangodb",
"--user=",
"--principal=pri",
"--role=arangodb",
"--mode=cluster",
"--async_replication=true",
"--minimal_resources_agent=mem(*):512;cpus(*):0.25;disk(*):512",
"--minimal_resources_dbserver=mem(*):512;cpus(*):0.25;disk(*):1024",
"--minimal_resources_secondary=mem(*):512;cpus(*):0.25;disk(*):1024",
"--minimal_resources_coordinator=mem(*):512;cpus(*):0.25;disk(*):1024",
"--nr_agents=1",
"--nr_dbservers=2",
"--nr_coordinators=2",
"--failover_timeout=86400",
"--arangodb_image=arangodb/arangodb-mesos:3.1",
"--secondaries_with_dbservers=false",
"--coordinators_with_dbservers=false"
],
"container":{
"type":"DOCKER",
"docker":{
"image":"arangodb/arangodb-mesos-framework:3.1",
"network":"HOST"
}
},
"healthChecks":[
{
"protocol":"HTTP",
"path":"/framework/v1/health.json",
"gracePeriodSeconds":3,
"intervalSeconds":10,
"portIndex":0,
"timeoutSeconds":10,
"maxConsecutiveFailures":0
}
]
}
Carefullyreviewthesettings(especiallytheIPsandtheresources).ThenyoucandeploytoM arathon:
curl-XPOST-H"Content-Type:application/json"http://url-of-marathon/v2/apps-d@arangodb3.json
AlternativelyusethewebinterfaceofM arathontodeployArangoDB.IthasaJSONmodeandyoucanusetheaboveconfigurationfile.
DeinstallationviaMarathon
AswithDC/OSyoufirstneedtoproperlycleanupanystateleftovers.
TheeasiestistosimplydeleteArangoDBandthendeploythecleanup-framework(seesectionarangodb-cleanup-framework).
Configurationoptions
TheArangodbM esosframeworkhasatonofdifferentoptionswhicharelistedanddescribedhere:
https://github.com/arangodb/arangodb-mesos-framework/tree/3.1
301
Cluster:Generic&Docker
AutomaticnativeClusters
SimilarlytohowtheM esosframeworkarangesanArangoDBclusterinaDC/OSenvironmentforyou, arangodbcandothisforyouin
aplainenvironment.
Byinvokingthefirst arangodbyoulaunchaprimarynode.Itwillbindanetworkport,andoutputthecommandsyouneedto
cut'n'pasteintotheothernodes.Let'sreviewtheprocessofsuchastartuponthreehostsnamed h01, h02,and h03:
arangodb@h01~>arangodb--ownAddressh01:4000
2017/06/1214:59:38Startingarangodbversion0.5.0+git,build5f97368
2017/06/1214:59:38ServingasmasterwithID'52698769'onh01:4000...
2017/06/1214:59:38Waitingfor3serverstoshowup.
2017/06/1214:59:38Usethefollowingcommandstostartotherservers:
arangodb--dataDir=./db2--joinh01:4000
arangodb--dataDir=./db3--joinh01:4000
2017/06/1214:59:38Listeningon0.0.0.0:4000(h01:4000)
Soyoucutthelines arangodb--data.dir=./db2--starter.join127.0.0.1andexecutethemfortheothernodes.Ifyourunitonanother
nodeonyournetwork,replacethe --starter.join127.0.0.1bythepublicIPofthefirsthost.
arangodbh02~>arangodb--dataDir=./db2--joinh01:4000
2017/06/1214:48:50Startingarangodbversion0.5.0+git,build5f97368
2017/06/1214:48:50Contactingmasterh01:4000...
2017/06/1214:48:50Waitingfor3serverstoshowup...
2017/06/1214:48:50Listeningon0.0.0.0:4000(:4000)
arangodbh03~>arangodb--dataDir=./db3--joinh01:4000
2017/06/1214:48:50Startingarangodbversion0.5.0+git,build5f97368
2017/06/1214:48:50Contactingmasterh01:4000...
2017/06/1214:48:50Waitingfor3serverstoshowup...
2017/06/1214:48:50Listeningon0.0.0.0:4000(:4000)
Oncethetwootherprocessesjoinedthecluster,andstartedtheirArangoDBserverprocesses(thismaytakeawhiledependingonyour
system),itwillinformyouwheretoconnecttheClusterfromaBrowser,shelloryourprogramm:
...
2017/06/1214:55:21coordinatorupandrunning.
Atthispointyoumayaccessyourclusterateithercoordinatorendpoint,http://h01:4002/,http://h02:4002/orhttp://h03:4002/.
AutomaticnativelocaltestClusters
Ifyouonlywantalocaltestcluster,youcanrunasinglestarterwiththe --starter.localargument.Itwillstarta3"machine"cluster
onyourlocalPC.
arangodb--starter.local
Note.AlocalclusterisintendedonlyfortestpurposessinceafailureofasinglePCwillbringdowntheentirecluster.
AutomaticDockerClusters
ArangoDBStartercanalsobeusedtolaunchclustersbasedondockercontainers.Itsabitmorecomplicated,sinceyouneedtoprovide
informationaboutyourenvironmentthatcan'tbeautodetected.
IntheDockerworldyouneedtotakecareaboutwherepersistantdataisstored,sincecontainersareintendedtobevolatile.Weusea
volumenamed arangodb1here:
302
Cluster:Generic&Docker
dockervolumecreatearangodb1
(Youcanuseanytypeofdockervolumethatfitsyoursetupinstead.)
WethenneedtodeterminethetheIPofthedockerhostwhereyouintendtorunArangoDBstarteron.Dependingonyouroperating
systemexecute ipaddr,ifconfigoripconfigtodetermineyourlocalipaddress.
192.168.1.0/24deveth0protokernelscopelinksrc192.168.1.32
SothisexampleusestheIP 192.168.1.32:
dockerrun-it--name=adb1--rm-p8528:8528\
-varangodb1:/data\
-v/var/run/docker.sock:/var/run/docker.sock\
arangodb/arangodb-starter\
--starter.address=192.168.1.32
Itwillstartthemasterinstance,andcommandyoutostarttheslaveinstances:
Unabletofindimage'arangodb/arangodb-starter:latest'locally
latest:Pullingfromarangodb/arangodb-starter
Digest:sha256:b87d20c0b4757b7daa4cb7a9f55cb130c90a09ddfd0366a91970bcf31a7fd5a4
Status:Downloadednewerimageforarangodb/arangodb-starter:latest
2017/06/1213:26:14Startingarangodbversion0.7.1,buildf128884
2017/06/1213:26:14ServingasmasterwithID'46a2b40d'on192.168.1.32:8528...
2017/06/1213:26:14Waitingfor3serverstoshowup.
2017/06/1213:26:14Usethefollowingcommandstostartotherservers:
dockervolumecreatearangodb2&&\
dockerrun-it--name=adb2--rm-p8533:8528-varangodb2:/data\
-v/var/run/docker.sock:/var/run/docker.sockarangodb/arangodb-starter:0.7\
--starter.address=192.168.1.32--starter.join=192.168.1.32
dockervolumecreatearangodb3&&\
dockerrun-it--name=adb3--rm-p8538:8528-varangodb3:/data\
-v/var/run/docker.sock:/var/run/docker.sockarangodb/arangodb-starter:0.7\
--starter.address=192.168.1.32--starter.join=192.168.1.32
Onceyoustarttheotherinstances,itwillcontinuelikethis:
2017/05/1109:05:45Addedmaster'fc673b3b':192.168.1.32,portOffset:0
2017/05/1109:05:45Addednewpeer'e98ea757':192.168.1.32,portOffset:5
2017/05/1109:05:50Addednewpeer'eb01d0ef':192.168.1.32,portOffset:10
2017/05/1109:05:51Startingservice...
2017/05/1109:05:51Lookingforarunninginstanceofagentonport8531
2017/05/1109:05:51Startingagentonport8531
2017/05/1109:05:52Lookingforarunninginstanceofdbserveronport8530
2017/05/1109:05:52Startingdbserveronport8530
2017/05/1109:05:53Lookingforarunninginstanceofcoordinatoronport8529
2017/05/1109:05:53Startingcoordinatoronport8529
2017/05/1109:05:58agentupandrunning(version3.2.devel).
2017/05/1109:06:15dbserverupandrunning(version3.2.devel).
2017/05/1109:06:31coordinatorupandrunning(version3.2.devel).
Andatleastittellsyouwhereyoucanworkwithyourcluster:
2017/05/1109:06:31Yourclustercannowbeaccessedwithabrowserat`http://192.168.1.32:8529`or
2017/05/1109:06:31using`arangosh--server.endpointtcp://192.168.1.32:8529`.
Underthehood
Thefirst arangodbyouran(asshownabove)willbecomethemasterinyoursetup,the --starter.joinwillbetheslaves.
303
Cluster:Generic&Docker
ThemasterdetermineswhichArangoDBserverprocessestolaunchonwhichslave,andhowtheyshouldcommunicate.Itwillthen
launchtheserverprocessesandmonitorthem.Onceithasdetectedthatthesetupiscompleteyouwillgettheprompt.Themasterwill
savethesetupforsubsequentstarts.
M orecomplicatedsetupoptionscanbefoundinArangoDBStartersReadme.
304
AdvancedTopics
AdvancedTopics
Incontrasttotheothertopicsinthischapterthatstrivetogetyousimplysetupinpreparedenvironments,Thefollowingchapters
describewhatsgoingonunderthehoodindetails,thecomponentsofArangoDBClusters,andhowthey'reputtogether:
Runningalocaltestsetup
Runningadistributedsetup
RunninginDocker
305
StandaloneAgency
LaunchingArangoDB'sstandalone"agency"
MultipleArangoDBinstancescanbedeployedasafault-tolerantdistributedstate
machine.
Whatisafault-tolerantstatemachineinthefirstplace?
Inmanyservicedeploymentsconsistingofarbitrarycomponentsdistributedovermultiplemachinesoneisfacedwiththechallengeof
creatingadependablecentralisedknowledgebaseorconfiguration.Implementationofsuchaserviceturnsouttobeoneofthemost
fundamentalproblemsininformationengineering.Whileitmayseemasiftherealisationofsuchaserviceiseasilyconceivable,
dependablityformulatesaparadoxononcomputernetworksperse.Ontheonehand,oneneedsadistributedsystemtoavoidasingle
pointoffailure.Ontheotherhand,onehastoestablishconsensusamongthecomputersinvolved.
Consensusisthekeywordhereanditsrealisationonanetworkprovestobefarfromtrivial.M anypapersandconferenceproceedings
havediscussedandevaluatedthiskeychallenge.Twoalgorithms,historicallyfarapart,havebecomewidelypopular,namelyPaxosand
itsderivativesandRaft.Discussingthemandtheirdifferences,althoughhighlyenjoyable,mustremainfarbeyondthescopeofthis
document.Findthereferencestothemainpublicationsatthebottomofthispage.
AtArangoDB,wedecidedtoimplementRaftasitisarguablytheeasiertounderstandandthusimplement.Insimpleterms,Raft
guaranteesthatalinearstreamoftransactions,isreplicatedinrealtimeamongagroupofmachinesthroughanelectedleader,whointurn
musthaveaccesstoandprojectleadershipuponanoverallmajorityofparticipatinginstances.InArangoDBweliketocalltheentirety
ofthecomponentsofthereplicatedtransactionlog,thatisthemachinesandtheArangoDBinstances,whichconstitutethereplicatedlog,
theagency.
Startup
Theagencymustconsistsofanoddnumberofagentsinordertobeabletoestablishanoverallmajorityandsomemeansfortheagentsto
beabletofindoneanotheratstartup.
Themostobviouswaywouldbetoinformallagentsoftheaddressesandportsoftherest.Thishowever,ismoreinformationthan
needed.Forexample,itwouldsuffice,ifallagentswouldknowtheaddressandportofthenextagentinacyclicfashion.Another
straitforwardsolutionwouldbetoinformallagentsoftheaddressandportofsaythefirstagent.
Clearlyallcases,whichwouldformdisjunctsubsetsofagentswouldbreakorintheleastimpairthefunctionalityoftheagency.From
thereontheagentswillgossipthemissinginformationabouttheirpeers.
Typically,oneachievesfairlyhighfault-tolerancewithlow,oddnumberofagentswhilekeepingthenecessarynetworktrafficata
minimum.Itseemsthatthetypicalagencysizewillbeinrangeof3to7agents.
Thebelowcommandsstartupa3-hostagencyononephysical/logicalboxwithports8529,8530and8531fordemonstrationpurposes.
Theadressofthefirstinstance,port8529,isknowntotheothertwo.Afteratmost2roundsofgossipping,thelast2agentswillhavea
completepictureoftheirsurroundingandpersistitforthenextrestart.
./build/bin/arangod--agency.activatetrue--agency.size3--agency.my-addresstcp://localhost:8529--server.authenticationfal
se--server.endpointtcp://0.0.0.0:8529agency-8529
./build/bin/arangod--agency.activatetrue--agency.size3--agency.endpointtcp://localhost:8529--agency.my-addresstcp://loc
alhost:8530--server.authenticationfalse--server.endpointtcp://0.0.0.0:8530agency-8530
./build/bin/arangod--agency.activatetrue--agency.size3--agency.endpointtcp://localhost:8529--agency.my-addresstcp://loc
alhost:8531--server.authenticationfalse--server.endpointtcp://0.0.0.0:8531agency-8531
Theparameter agency.endpointisthekeyingredientforthesecondandthirdinstancestofindthefirstinstanceandthusforma
completeagency.Pleaserefertothetheshell-script scripts/startStandaloneAgency.shongithuborinthesourcedirectory.
Key-value-storeAPI
Theagencyshouldbeupandrunningwithinacoupleofseconds,duringwhichtheinstanceshavegossipedtheirwayintoknowingthe
otheragentsandelectedaleader.ThepublicAPIcanbecheckedforthestateoftheconfiguration:
curl-slocalhost:8529/_api/agency/config
306
StandaloneAgency
{
"term":1,
"leaderId":"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98",
"lastCommitted":1,
"lastAcked":{
"ac129027-b440-4c4f-84e9-75c042942171":0.21,
"c54dbb8a-723d-4c82-98de-8c841a14a112":0.21,
"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98":0
},
"configuration":{
"pool":{
"ac129027-b440-4c4f-84e9-75c042942171":"tcp://localhost:8531",
"c54dbb8a-723d-4c82-98de-8c841a14a112":"tcp://localhost:8530",
"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98":"tcp://localhost:8529"
},
"active":[
"ac129027-b440-4c4f-84e9-75c042942171",
"c54dbb8a-723d-4c82-98de-8c841a14a112",
"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98"
],
"id":"f5d11cde-8468-4fd2-8747-b4ef5c7dfa98",
"agencysize":3,
"poolsize":3,
"endpoint":"tcp://localhost:8529",
"minping":0.5,
"maxping":2.5,
"supervision":false,
"supervisionfrequency":5,
"compactionstepsize":1000,
"supervisiongraceperiod":120
}
}
Tohighlightsomedetailsintheaboveoutputlookfor "term"and "leaderId".Botharekeyinformationaboutthecurrentstateofthe
Raftalgorithm.Youmayhavenotedthatthefirstelectiontermhasestablishedarandomleaderfortheagency,whoisinchargeof
replicationofthestatemachineandforallexternalreadandwriterequestsuntilsuchtimethattheprocessgetsisolatedfromtheother
twosubsequenctlylosingitsleadership.
ReadandWriteAPIs
Generally,allreadandwriteaccessesaretransactionsmoreoveranyreadandwriteaccessmayconsistofmultiplesuchtransactions
formulatedasarraysofarraysinJSONdocuments.
Readtransaction
Anagencystartedfromscratchwilldealwiththesimplestqueryasfollows:
curl-Llocalhost:8529/_api/agency/read-d'[["/"]]'
[{}]
Theaboverequestforanemptykeyvaluestorewillreturnwithanemptydocument.Theinnerarraybracketswillaggregatearesult
frommultiplesourcesinthekey-value-storewhiletheouterarraywilldelivermultiplesuchaggregatedresults.Alsonotethe -Lcurl
flag,whichallowstherequesttofollowredirectstothecurrentleader.
Considerthefollowingkey-value-store:
{
"baz":12,
"corge":{
"e":2.718281828459045,
"pi":3.14159265359
},
"foo":{
"bar":"HelloWorld"
},
307
StandaloneAgency
"qux":{
"quux":"HelloWorld"
}
}
Thefollowingarrayofreadtransactionswillyield:
curl-Llocalhost:8529/_api/agency/read-d'[["/foo","/foo/bar","/baz"],["/qux"]]'
[
{
"baz":12,
"foo":{
"bar":"HelloWorld"
}
},
{
"qux":{
"quux":"HelloWorld"
}
}
]
Notethattheresultisanarrayoftworesultsforthefirstandsecondreadtransactionsfromaboveaccordingly.Alsonotethattheresults
fromthefirstreadtransactionareaggregatedinto
{
"baz":12,
"foo":{
"bar":"HelloWorld"
}
}
Theaggregationisperformedon2levels:
1.
/foo/bariseliminatedasasubsetof /foo
2. Theresultsfrom /fooand /bararejoined
Thewordtransactionmeansherethatitisguaranteedthatallaggregationshappeninquasi-realtimeandthatnowriteaccesscouldhave
happenedinbetween.
Btw,thesametransactiononthevirginkey-valuestorewouldproduce [{},{}]
WriteAPI:
ThewriteAPImustunfortunatelybealittlemorecomplex.M ultipleroadsleadtoRome:
curl-Llocalhost:8529/_api/agency/write-d'[[{"/foo":{"op":"push","new":"bar"}}]]'
curl-Llocalhost:8529/_api/agency/write-d'[[{"/foo":{"op":"push","new":"baz"}}]]'
curl-Llocalhost:8529/_api/agency/write-d'[[{"/foo":{"op":"push","new":"qux"}}]]'
and
curl-Llocalhost:8529/_api/agency/write-d'[[{"foo":["bar","baz","qux"]}]]'
areequivalentforexampleandwillcreateandfillanarrayat /foo.Here,again,theoutermostarrayisthecontainerforthetransaction
arrays.
WedocumententacompleteguideoftheAPIintheAPIsection.
308
Cluster:Localtestsetups
LaunchinganArangoDBclusterfortesting
AnArangoDBclusterconsistsofseveralrunningtaskswhichformthecluster.ArangoDBitselfwon'tstartormonitoranyofthesetasks.
Soitwillneedsomekindofsupervisorwhichismonitoringandstartingthesetasks.ForproductionusagewerecommendusingApache
M esosastheclustersupervisor.
HoweverstartingaclustermanuallyispossibleandisaveryeasymethodtogetafirstimpressionofwhatanArangoDBclusterlooks
like.
Theeasiestwaytostartalocalclusterfortestingpurposesistorun scripts/startLocalCluster.shfromacloneofthesource
repositoryaftercompilingArangoDBfromsource(seeinstructionsinthefile README_maintainers.mdintherepository.Thiswillstart1
Agency,2DBServersand1Coordinator.Tostoptheclusterissue scripts/stopLocalCluster.sh.
ThissectionwilldiscusstherequiredparametersforeveryroleinanArangoDBcluster.BesuretoreadtheArchitecturedocumentation
togetabasicunderstandingoftheunderlyingarchitectureandtheinvolvedrolesinanArangoDBcluster.
Inthefollowingsectionswewillgothroughtherelevantoptionsperrole.
Agency
Tostartupanagencyyoufirsthavetoactivateit.Thisisdonebyproviding --agency.activatetrue.
Tostartuptheagencyinitsfaulttolerantmodesetthe --agency.sizeto 3.Youwillthenhavetoprovideatleast3agentsbeforethe
agencywillstartoperation.
Duringinitializationtheagentshavetofindeachother.Todosoprovideatleastonecommon --agency.endpoint.Theagentswillthen
coordinatestartupthemselves.Theywillannouncethemselveswiththeirexternaladdresswhichmaybespecifiedusing --agency.myaddress.ThisisrequiredinbridgeddockersetupsorNATedenvironments.
Soinsummarythisiswhatyourstartupmightlooklike:
309
Cluster:Localtestsetups
arangod--server.endpointtcp://0.0.0.0:5001--agency.my-address=tcp://127.0.0.1:5001--server.authenticationfalse--agency.ac
tivatetrue--agency.size3--agency.endpointtcp://127.0.0.1:5001--agency.supervisiontrue--database.directoryagency1&
arangod--server.endpointtcp://0.0.0.0:5002--agency.my-address=tcp://127.0.0.1:5002--server.authenticationfalse--agency.ac
tivatetrue--agency.size3--agency.endpointtcp://127.0.0.1:5001--agency.supervisiontrue--database.directoryagency2&
arangod--server.endpointtcp://0.0.0.0:5003--agency.my-address=tcp://127.0.0.1:5003--server.authenticationfalse--agency.ac
tivatetrue--agency.size3--agency.endpointtcp://127.0.0.1:5001--agency.supervisiontrue--database.directoryagency3&
Ifyouarehappywithasingleagent,thensimplyuseasinglecommandlikethis:
arangod--server.endpointtcp://0.0.0.0:5001--server.authenticationfalse--agency.activatetrue--agency.size1--agency.endp
ointtcp://127.0.0.1:5001--agency.supervisiontrue--database-directoryagency1&
Furthermore,inthefollowingsectionswhen --cluster.agency-addressisusedmultipletimestospecifyallthreeagentaddresses,just
useasingleoption --cluster.agency.addresstcp://127.0.0.1:5001instead.
CoordinatorsandDBServers
Thesetworolesshareacommonsetofrelevantoptions.Firstyoushouldspecifytheroleusing --cluster.my-role.Thiscaneitherbe
PRIMARY(adatabaseserver)or COORDINATOR.Bothneedsomeuniqueinformationwithwhichtheywillregisterintheagency,too.This
couldforexamplebesomecombinationofhostnameandportorwhateveryouhaveathand.Howeveritmustbeuniqueforeach
instanceandbeprovidedasvaluefor --cluster.my-local-info.Furthermoreprovidetheexternalendpoint(IPandport)ofthetaskvia
--cluster.my-address.
Thefollowingisafull-exampleofwhatitmightlooklike:
arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8529--cluster.my-addresstcp://127.0.0.1:8529--cluster.
my-local-infodb1--cluster.my-rolePRIMARY--cluster.agency-endpointtcp://127.0.0.1:5001--cluster.agency-endpointtcp://127.
0.0.1:5002--cluster.agency-endpointtcp://127.0.0.1:5003--database.directoryprimary1&
arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8530--cluster.my-addresstcp://127.0.0.1:8530--cluster.
my-local-infodb2--cluster.my-rolePRIMARY--cluster.agency-endpointtcp://127.0.0.1:5001--cluster.agency-endpointtcp://127.
0.0.1:5002--cluster.agency-endpointtcp://127.0.0.1:5003--database.directoryprimary2&
arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8531--cluster.my-addresstcp://127.0.0.1:8531--cluster.
my-local-infocoord1--cluster.my-roleCOORDINATOR--cluster.agency-endpointtcp://127.0.0.1:5001--cluster.agency-endpointtcp
://127.0.0.1:5002--cluster.agency-endpointtcp://127.0.0.1:5003--database.directorycoordinator&
Noteinparticularthattheendpointdescriptionsgivenunder --cluster.my-addressand --cluster.agency-endpointmustnotusethe
IPaddress 0.0.0.0becausetheymustcontainanactualaddressthatcanberoutedtothecorrespondingserver.The 0.0.0.0in -server.endpointsimplymeansthattheserverbindsitselftoallavailablenetworkdeviceswithallavailableIPaddresses.
UponregisteringwiththeagencyduringstartuptheclusterwillassignanIDtoeverytask.ThegeneratedIDwillbeprintedouttothe
logorcanbeaccessedviathehttpAPIbycalling http://server-address/_admin/server/id.Shouldyoueverhavetorestartatask,
simplyreusethesamevaluefor --cluster.my-local-infoandthesameIDwillbepicked.
YouhavenowlaunchedacompleteArangoDBclusterandcancontactitscoordinatorattheendpoint tcp://127.0.0.1:8531,which
meansthatyoucanreachthewebUIunder http://127.0.0.1:8531.
310
Cluster:Processes
LaunchinganArangoDBclusteronmultiplemachines
Essentially,onecanusethemethodfromtheprevioussectiontostartanArangoDBclusteronmultiplemachinesaswell.Theonly
changesarethatonehastoreplacealllocaladdresses 127.0.0.1bytheactualIPaddressofthecorrespondingserver.
IfweassumethatyouwanttostartyouArangoDBclusteronthreedifferentmachineswithIPaddresses
192.168.1.1
192.168.1.2
192.168.1.3
thenthecommandsyouhavetouseare(youcanusehostnamesiftheycanberesolvedtoIPaddressesonallmachines):
On192.168.1.1:
sudoarangod--server.endpointtcp://0.0.0.0:5001--agency.my-addresstcp://192.168.1.1:5001--server.authenticationfalse--ag
ency.activatetrue--agency.size3--agency.supervisiontrue--database.directoryagency
On192.168.1.2:
sudoarangod--server.endpointtcp://0.0.0.0:5001--agency.my-addresstcp://192.168.1.2:5001--server.authenticationfalse--ag
ency.activatetrue--agency.size3--agency.supervisiontrue--database.directoryagency
On192.168.1.3:
sudoarangod--server.endpointtcp://0.0.0.0:5001--agency.my-addresstcp://192.168.1.3:5001--server.authenticationfalse--ag
ency.activatetrue--agency.size3--agency.endpointtcp://192.168.1.1:5001--agency.endpointtcp://192.168.1.2:5001--agency.e
ndpointtcp://192.168.1.3:5001--agency.supervisiontrue--database.directoryagency
On192.168.1.1:
sudoarangod--server.authentication=false--server.endpointtcp://0.0.0.0:8529--cluster.my-addresstcp://192.168.1.1:8529--c
luster.my-local-infodb1--cluster.my-rolePRIMARY--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agency-endpointt
cp://192.168.1.2:5001--cluster.agency-endpointtcp://192.168.1.3:5001--database.directoryprimary1&
On192.168.1.2:
sudoarangod--server.authentication=false--server.endpointtcp://0.0.0.0:8530--cluster.my-addresstcp://192.168.1.2:8530--c
luster.my-local-infodb2--cluster.my-rolePRIMARY--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agency-endpointt
cp://192.168.1.2:5001--cluster.agency-endpointtcp://192.168.1.3:5001--database.directoryprimary2&
On192.168.1.3:
arangod--server.authentication=false--server.endpointtcp://0.0.0.0:8531--cluster.my-addresstcp://192.168.1.3:8531--cluste
r.my-local-infocoord1--cluster.my-roleCOORDINATOR--cluster.agency-endpointtcp://192.168.1.1:5001--cluster.agency-endpoint
tcp://192.168.1.2:5001--cluster.agency-endpointtcp://192.168.1.3:5001--database.directorycoordinator&
Obviously,itwouldnolongerbenecessarytousedifferentportnumbersondifferentservers.Wehavechosentokeepallportnumbers
incomparisontothelocalsetuptominimizethenecessarychanges.
Afterhavingswallowedtheselongishcommands,wehopethatyouappreciatethesimplicityofthesetupwithApacheM esosand
DC/OS.
311
Cluster:Docker
ArangoDBClusterandDocker
Networking
AbitofextracarehastobeinvestedduetothewayinwhichDockerisolatesitsnetwork.Bydefaultitfullyisolatesthenetworkandby
doingsoanendpointlike --server.endpointtcp://0.0.0.0:8529willonlybindtoallinterfacesinsidetheDockercontainerwhichdoes
notincludeanyexternalinterfaceonthehostmachine.Thismaybesufficientifyoujustwanttoaccessitlocallybutincaseyouwantto
exposeittotheoutsideyoumustfacilitateDockersportforwardingusingthe -pcommandlineoption.Besuretochecktheofficial
Dockerdocumentation.
Tosimplymakearangodbavailableonallhostinterfacesonport8529:
dockerrun-p8529:8529-eARANGO_NO_AUTH=1arangodb
AnotherpossibilityistostartDockervianetworkmode host.Thisispossiblebutgenerallynotrecommended.Todoitanywaycheck
theDockerdocumentationfordetails.
DockerandClustertasks
TostarttheclusterviaDockerisbasicallythesameasstartinglocallyoronmultiplemachines.Howeverjustlikewiththesingle
networkingimagewewillfacenetworkingissues.Youcansimplyusethe -pflagtomaketheindividualtaskavailableonthehost
machineoryoucoulduseDocker'slinkstoenabletaskintercommunication.
PleasenotethattherearesomeflagsthatspecifyhowArangoDBcanreachataskfromtheoutside.Theseareveryimportantandbuilt
forthisexactusecase.Anexampleconfigurationmightlooklikethis:
dockerrun-eARANGO_NO_AUTH=1-p192.168.1.1:10000:8529arangodb/arangodbarangod--server.endpointtcp://0.0.0.0:8529--clust
er.my-addresstcp://192.168.1.1:10000--cluster.my-local-infodb1--cluster.my-rolePRIMARY--cluster.agency-endpointtcp://192
.168.1.1:5001--cluster.agency-endpointtcp://192.168.1.2:5002--cluster.agency-endpointtcp://192.168.1.3:5003
ThiswillstartaprimaryDBserverwithinaDockercontainerwithanisolatednetwork.WithintheDockercontaineritwillbindtoall
interfaces(thiswillbe127.0.0.1:8529andsomeinternalDockeriponport8529).Bysupplying -p192.168.1.1:10000:8529weare
establishingaportforwardingfromourlocalIP(192.168.1.1port10000inthisexample)toport8529insidethecontainer.Withinthe
commandwearetellingarangodhowitcanbereachedfromtheoutside --cluster.my-addresstcp://192.168.1.1:10000.This
informationwillbeforwardedtotheagencysothattheothertasksinyourclustercanseehowthisparticularDBServermaybereached.
312
Administration
Administration
M ostadministrationcanbemanagedusingthearangosh.
Filesystems
Asonewouldexpectforadatabase,werecommendalocallymountedfilesystems.
NFSorsimilarnetworkfilesystemswillnotwork.
OnLinuxwerecommendtheuseofext4fs,onWindowsNTFSandonM acOSHFS+.
WerecommendtonotuseBTRFSonLinux.ItisknowntonotworkwellinconjunctionwithArangoDB.Weexperiencedthat
ArangoDBfaceslatencyissuesonaccessingitsdatabasefilesonBTRFSpartitions.InconjunctionwithBTRFSandAUFSwealsosaw
datalossonrestart.
313
WebInterface
WebInterface
ArangoDBcomeswithabuilt-inwebinterfaceforadministration.Theinterfacediffersforstandaloneinstancesandclustersetups.
Standalone:
Cluster:
314
WebInterface
315
Dashboard
Dashboard
TheDashboardtabprovidesstatisticswhicharepolledregularlyfromtheArangoDBserver.
RequestsStatistics:
Requestspersecond
Requesttypes
Numberofclientconnections
Transfersize
Transfersize(distribution)
Averagerequesttime
Averagerequesttime(distribution)
SystemResources:
Numberofthreads
M emory
Virtualsize
M ajorpagefaults
UsedCPUtime
Replication:
Replicationstate
Totals
Ticks
Progress
316
Cluster
Cluster
Theclustersectiondisplaysstatisticsaboutthegeneralclusterperformance.
Statistics:
Availableandmissingcoordinators
Availableandmissingdatabaseservers
M emoryusage(percent)
Currentconnections
Data(bytes)
HTTP(bytes)
Averagerequesttime(seconds)
Nodes
Overview
Theoverviewshowsavailableandmissingcoordinatorsanddatabaseservers.
317
Cluster
Functions:
CoordinatorDashboard:ClickonaCoordinatorwillopenastatisticsdashboard.
Information(Coordinator/Databaseservers):
Name
Endpoint
LastHeartbeat
Status
Health
Shards
Theshardsectiondisplaysallavailableshardedcollections.
318
Cluster
Functions:
M oveShardLeader:Clickonaleaderdatabaseofashardserverwillopenamovesharddialog.Shardscanbetransferredtoall
availabledatabasservers,excepttheleadingdatabaseserveroranavailablefollower.
M oveShardFollower:Clickonafollowerdatabaseofashardwillopenamovesharddialog.Shardscanbetransferredtoall
availabledatabasservers,excepttheleadingdatabaseserveroranavailablefollower.
RebalanceShards:Anewdatabaseserverwillnothaveanyshards.Withtherebalancefunctionalitytheclusterwillstarttorebalance
shardsincludingemptydatabaseservers.
Information(collection):
Shard
Leader(greenstate:synciscomplete)
Followers
319
Collections
Collections
Thecollectionssectiondisplaysallavailablecollections.Fromhereyoucancreatenewcollectionsandjumpintoacollectionfordetails
(clickonacollectiontile).
Functions:
A:Togglefilterproperties
B:Searchcollectionbyname
D:Createcollection
C:Filterproperties
H:Showcollectiondetails(clicktile)
Information:
E:Collectiontype
F:Collectionstate(unloaded,loaded,...)
G:Collectionname
Collection
320
Collections
Therearefourviewcategories:
1. Content:
Createadocument
Deleteadocument
Filterdocuments
Downloaddocuments
Uploaddocuments
2. Indices:
Createindices
Deleteindices
3. Info:
Detailedcollectioninformationandstatistics
4. Settings:
Configurename,journalsize,indexbuckets,waitforsync
Deletecollection
Truncatecollection
Unload/Loadcollection
Savemodifedproperties(name,journalsize,indexbuckets,waitforsync)
Additionalinformation:
Uploadformat:
I.Line-wise
{"_key":"key1",...}
{"_key":"key2",...}
II.JSONdocumentsinalist
[
{"_key":"key1",...},
{"_key":"key2",...}
321
Collections
]
322
Document
Document
Thedocumentsectionoffersaeditorwhichletyoueditdocumentsandedgesofacollection.
Functions:
Editdocument
Savedocument
Deletedocment
SwitchbetweenTree/Code-M ode
Createanewdocument
Information:
Displays:_id,_rev,_keyproperties
323
Queries
QueryView
Thequeryviewoffersyouthreedifferentsubviews:
Editor
RunningQueries
SlowQueryHistory
AQLQueryEditor
ThewebinterfaceoffersaAQLQueryEditor:
Theeditorissplitintotwoparts,thequeryeditorpaneandthebindparameterpane.
Theleftpaneisyourregularqueryinputfield,whereyoucaneditandthenexecuteorexplainyourqueries.Bydefault,theenteredbind
parameterwillautomaticallyberecognizedandshowninthebindparametertableintherightpane,whereyoucaneasilyeditthem.
Theinputfieldsareequippedwithtypedetection.Thismeansyoudon'thavetousequotemarksaroundstring,justwritethemas-is.
Numberswillbetreatedasnumbers,trueandfalseasbooleans,nullasnull-typevalue.Squarebracketscanbeusedtodefinearrays,and
curlybracesforobjects(keysandvalueshavetobesurroundedbydoublequotes).Thiswillmostlybewhatyouwant.Butifyouwant
toforcesomethingtobetreatedasstring,usequotationmarksforthevalue:
123//interpretedasnumber
"123"//interpretedasstring
["foo","bar",123,true]//interpretedasarray
['foo','bar',123,true]//interpretedasstring
324
Queries
IfyouareusedtoworkwithJSON,youmaywanttoswitchthebindparametereditortoJSONmodebyclickingontheupperright
togglebutton.YoucantheneditthebindparametersinrawJSONformat.
CustomQueries
TosavethecurrentqueryusetheSavebuttoninthetopleftcorneroftheeditororusetheshortcut(seebelow).
BypressingtheQueriesbuttoninthetopleftcorneroftheeditoryouactivatethecustomqueriesview.Hereyoucanselectapreviously
storedcustomqueryoroneofourqueryexamples.
Clickonaquerytitletogetacodepreview.Inaddition,thereareactionbuttonsto:
Copytoeditor
Explainquery
Runquery
Deletequery
Forthebuilt-inexamplequeries,thereisonlyCopytoeditoravailable.
ToexportorimportqueriestoandfromJSONyoucanusethebuttonsontheright-handside.
Result
325
Queries
Eachqueryyouexecuteorexplainopensupanewresultbox,soyouareabletofireupmultiplequeriesandviewtheirresultsatthe
sametime.Everyqueryresultboxgivesyoudetailedqueryinformationandofcoursethequeryresultitself.Theresultboxescanbe
dismissedindividually,oraltogetherusingtheRemoveresultsbutton.Thetogglebuttoninthetoprightcornerofeachboxswitchesback
andforthbetweentheResultandAQLquerywithbindparameters.
Spotlight
Thespotlightfeatureopensupamodalview.ThereyoucanfindallAQLkeywords,AQLfunctionsandcollections(filteredbytheir
type)tohelpyoutobemoreproductiveinwritingyourqueries.Spotlightcanbeopenedbythemagicwandiconinthetoolbarorvia
shortcut(seebelow).
AQLEditorShortcuts
326
Queries
Ctrl/Cmd+Returntoexecuteaquery
Ctrl/Cmd+Shift+Returntoexplainaquery
Ctrl/Cmd+Shift+Stosavethecurrentquery
Ctrl/Cmd+Shift+Ctotogglecomments
Ctrl+Spacetoopenupthespotlightsearch
Ctrl+Cmd+Ztoundolastchange
Ctrl+Cmd+Shift+Ztoredolastchange
RunningQueries
TheRunningQueriestabgivesyouacompactoverviewofallrunningqueries.Byclickingtheredminusbutton,youcanabortthe
executionofarunningquery.
SlowQueryHistory
327
Queries
TheSlowQueryHistorytabgivesyouacompactoverviewofallpastslowqueries.
328
Graphs
Graphs
TheGraphstabprovidesaviewerfacilityforgraphdatastoredinArangoDB.ItallowsbrowsingArangoDBgraphsstoredinthe
_graphssystemcollectionoragraphconsistingofanarbitraryvertexandedgecollection.
Pleasenotethatthegraphviewerrequirescanvas(optional:webgl)supportinyourbrowser.EspeciallyInternetExplorerbrowsersolder
thanversion9arelikelytonotsupportthis.
GraphViewer
329
Graphs
TopToolbarFunctions:
Loadfullgraph(Alsonodeswithoutconnectionswillbedrawn.Usefulduringgraphmodelingsetup)
Takeagraphscreenshot
Startfullscreenmode
Opengraphoptionsmenu
DefaultContextM enu(mouse-clickbackground):
Addanewnode
Closevisiblecontextmenu(s)
NodeContextM enu(mouse-clicknode):
Deletenode
Editnode
Expandnode(Showallboundedges)
Drawedge(Connectwithanothernode)
Setasstartnode(TheGraphwillrerenderstartingtheselectednodeandgivenoptions(graphoptionsmenu))
EdgeContextM enu(mouse-clickedge):
Editedge
Deleteedge
EdgeHighlighting(right-mouse-clicknode):
Highlightalledgesconnectedtothenode(right-clickatthebackgroundwillremovehighlighting)
330
Graphs
GraphViewerOptions
GraphOptionsM enu:
Startnode(string-validnodeidorspaceseperatedlistofid's):Heartofyourgraph.Renderingandtraversingwillstartfromhere.
Emptyvaluemeans:arandomstartingpointwillbeused.
Layout:Differentgraphlayoutingalgoritms.Nooverlap(optimal:biggraph),forcelayout(optimal:mediumgraph),fruchtermann
(optimal:littletomediumgraph).
Renderer:Canvasmodeallowsediting.WebGLcurrentlyoffersonlydisplaymode(alotfasterwithmuchnodes/edges).
Searchdepth(number):Searchdepthwhichisstartingfromyourstartnode.
Limit(number):Limitnodescount.Ifemptyorzero,nolimitisset.
NodesOptionsM enu:
Label(string):Nodeswillbelabeledbythisattribute.Ifnodeattributeisnotfound,nolabelwillbedisplayed.
AddCollectionName:Thisappendsthecollectionnametothelabel,ifitexists.
ColorByCollections:Shouldnodesbecolorizedbytheircollection?Ifenabled,nodecolorandnodecolorattributewillbeignored.
Color:Defaultnodecolor.
ColorAttribute(string):Ifanattributeisgiven,nodeswillthenbecolorizedbytheattribute.Thissettingignoresdefaultnodecolor
ifset.
SizeByConnections:Shouldnodesbesizedbytheiredgescount?Ifenabled,nodesizingattributewillbeignored.
SizingAttribute(number):Defaultnodesize.Numericvalue>0.
EdgesOptionsM enu:
Label(string):Edgeswillbelabeledbythisattribute.Ifedgeattributeisnotfound,nolabelwillbedisplayed.
AddCollectionName:Thisappendsthecollectionnametothelabel,ifitexists.
ColorByCollections:Shouldedgesbecolorizedbytheircollection?Ifenabled,edgecolorandedgecolorattributewillbeignored.
Color:Defaultedgecolor.
ColorAttribute(string):Ifanattributeisgiven,edgeswillthenbecolorizedbytheattribute.Thissettingignoresdefaultnodecolor
ifset.
Type:Therendereroffersmultipletypesofrendering.Theyonlydifferintheirdisplaystyle,exceptforthetype'curved'.The
curvedtypeallowstodisplaymorethanoneedgesbetweentwonodes.
331
Graphs
332
Services
Services
Theservicessectiondisplaysallinstalledfoxxapplications.Youcancreatenewservicesorgointoadetailedviewofachoosenservice.
CreateService
Therearefourdifferentpossibilitiestocreateanewservice:
1. Createserviceviazipfile
2. Createserviceviagithubrepository
3. CreateserviceviaofficialArangoDBstore
4. Createablankservicefromscratch
333
Services
ServiceView
Thissectionoffersseveralinformationaboutaspecificservice.
Therearefourviewcategories:
1. Info:
Displaysname,shortdescription,license,version,mode(production,development)
Offersabuttontogototheservicesinterface(ifavailable)
334
Services
2. Api:
DisplayAPIasSwaggerUI
DisplayAPIasRAWJSON
3. Readme:
Displaystheservicesmanual(ifavailable)
4. Settings:
Downloadserviceaszipfile
Runservicetests(ifavailable)
Runservicescripts(ifavailable)
Configuredependencies(ifavailable)
Changeserviceparameters(ifavailable)
Changemode(production,development)
Replacetheservice
Deletetheservice
335
Users
ManagingUsersintheWebInterface
ArangoDBusersaregloballystoredinthe_systemdatabaseandcanonlybemanangedwhileloggedontothisdatabase.Thereyoucan
findtheUserssection:
General
SelectausertobringuptheGeneraltabwiththeusername,nameandactivestatus,aswellasoptionstodeletetheuserorchangethe
password.
336
Users
Permissions
SelectauserandgotothePermissionstab.Youwillseealistofdatabasesandtheircorrespondingdatabaseaccesslevelforthatuser.
337
Users
Pleasenotethatserveraccesslevelfollowsfromtheaccesslevelonthedatabase_system.Furthermore,thedefaultdatabaseaccesslevel
forthisuserappearintheartificialrowwiththedatabasename *.
Belowthistableisanotheroneforthecollectioncategoryaccesslevels.Atfirst,itshowsthelistofdatabases,too.Ifyouclickona
database,thelistofcollectionsinthatdatabasewillbeopenandyoucanseethedefinedcollectionaccesslevelsforeachcollectionofthat
database(whichcanbeallunselectedwhichmeansthatnothingisexplicitlyset).Thedefaultaccesslevelsforthisuseranddatabase
appearintheartificialrowwiththecollectionname *.
AlsoseeManagingUsersaboutaccesslevels.
338
Logs
Logs
Thelogssectiondisplaysallavailablelogentries.Logentriesarefilterablebytheirlogleveltypes.
Functions:
Filterlogentriesbyloglevel(all,info,error,warning,debug)
Information:
Loglevel
Date
M essage
339
ArangoDBShell
ArangoDBShellIntroduction
TheArangoDBshell(arangosh)isacommand-linetoolthatcanbeusedforadministrationofArangoDB,includingrunningad-hoc
queries.
ThearangoshbinaryisshippedwithArangoDB.ItoffersaJavaScriptshellenvironmentprovidingaccesstotheArangoDBserver.
Arangoshcanbeinvokedlikethis:
unix>arangosh
BydefaultarangoshwilltrytoconnecttoanArangoDBserverrunningonserverlocalhostonport8529.Itwillusetheusernameroot
andanemptypasswordbydefault.Additionallyitwillconnecttothedefaultdatabase(_system).Allthesedefaultscanbechangedusing
thefollowingcommand-lineoptions:
--server.database:nameofthedatabasetoconnectto
--server.endpoint:endpointtoconnectto
--server.username:databaseusername
--server.password:passwordtousewhenconnecting
--server.authentication:whetherornottouseauthentication
Forexample,toconnecttoanArangoDBserveronIP192.168.173.13onport8530withtheuserfooandusingthedatabasetest,use:
unix>arangosh\
--server.endpointtcp://192.168.173.13:8530\
--server.usernamefoo\
--server.databasetest\
--server.authenticationtrue
arangoshwillthendisplayapasswordpromptandtrytoconnecttotheserverafterthepasswordwasentered.
Tochangethecurrentdatabaseaftertheconnectionhasbeenmade,youcanusethe db._useDatabase()commandinarangosh:
arangosh>db._createDatabase("myapp");
true
arangosh>db._useDatabase("myapp");
true
arangosh>db._useDatabase("_system");
true
arangosh>db._dropDatabase("myapp");
true
Togetalistofavailablecommands,arangoshprovidesahelp()function.Callingitwilldisplayhelpfulinformation.
arangoshalsoprovidesauto-completion.Additionalinformationonavailablecommandsandmethodsisthusprovidedbytypingthe
firstfewlettersofavariableandthenpressingthetabkey.Itisrecommendtotrythiswithenteringdb.(withoutpressingreturn)and
thenpressingtab.
Bytheway,arangoshprovidesthedbobjectbydefault,andthisobjectcanbeusedforswitchingtoadifferentdatabaseandmanaging
collectionsinsidethecurrentdatabase.
Foralistofavailablemethodsforthedbobject,type
arangosh>db._help();
showexecutionresults
youcanpastemultiplelinesintoarangosh,giventhefirstlineendswithanopeningbrace:
arangosh>for(vari=0;i<10;i++){
340
ArangoDBShell
........>require("@arangodb").print("Helloworld"+i+"!\n");
........>}
showexecutionresults
ToloadyourownJavaScriptcodeintothecurrentJavaScriptinterpretercontext,usetheloadcommand:
require("internal").load("/tmp/test.js")//<-Linux/MacOS
require("internal").load("c:\\tmp\\test.js")//<-Windows
Exitingarangoshcanbedoneusingthekeycombination <CTRL>+Dorbytyping quit<CR>
Escaping
InAQL,escapingisdonetraditionallywiththebackslashcharacter: \.Asseenabove,thisleadstodoublebackslasheswhenspecifying
Windowspaths.Arangoshrequiresanotherlevelofescaping,alsowiththebackslashcharacter.Itaddsuptofourbackslashesthatneed
tobewritteninArangoshforasingleliteralbackslash( c:\tmp\test.js):
db._query('RETURN"c:\\\\tmp\\\\test.js"')
Youcanusebindvariablestomitigatethis:
varsomepath="c:\\tmp\\test.js"
db._query(aql`RETURN${somepath}`)
341
ShellOutput
ArangoDBShellOutput
Bydefault,theArangoDBshellusesaprettyprinterwhenJSONdocumentsareprinted.Thisensuresdocumentsareprintedina
human-readableway:
arangosh>db._create("five")
arangosh>for(i=0;i<5;i++)db.five.save({value:i})
arangosh>db.five.toArray()
showexecutionresults
Whilethepretty-printerproducesnicelookingresults,itwillneedalotofscreenspaceforeachdocument.Sometimes,amoredense
outputmightbebetter.Inthiscase,theprettyprintercanbeturnedoffusingthecommandstop_pretty_print().
Toturnonprettyprintingagain,usethestart_pretty_print()command.
342
Configuration
ArangoDBShellConfiguration
arangoshwilllookforauser-definedstartupscriptnamed.arangosh.rcintheuser'shomedirectoryonstartup.Thehomedirectorywill
likelybe /home/<username>/onUnix/Linux,andisdeterminedonWindowsbypeekingintotheenvironmentvariables %HOMEDRIVE%and
%HOMEPATH%.
Ifthefile.arangosh.rcispresentinthehomedirectory,arangoshwillexecutethecontentsofthisfileinsidetheglobalscope.
Youcanusethistodefineyourownextravariablesandfunctionsthatyouneedoften.Forexample,youcouldputthefollowingintothe
.arangosh.rcfileinyourhomedirectory:
//"var"keywordavoidedintentionally...
//otherwise"timed"wouldnotsurvivethescopeofthisscript
global.timed=function(cb){
console.time("callback");
cb();
console.timeEnd("callback");
};
Thiswillmakeafunctionnamedtimedavailableinarangoshintheglobalscope.
Youcannowstartarangoshandinvokethefunctionlikethis:
timed(function(){
for(vari=0;i<1000;++i){
db.test.save({value:i});
}
});
Pleasekeepinmindthat,ifpresent,the.arangosh.rcfileneedstocontainvalidJavaScriptcode.Ifyouwantanyvariablesintheglobal
scopetosurviveyouneedtoomitthevarkeywordforthem.Otherwisethevariableswillonlybevisibleinsidethescriptitself,butnot
outside.
343
Details
DetailsabouttheArangoDBShell
Aftertheserverhasbeenstarted,youcanusetheArangoDBshell(arangosh)toadministratetheserver.Withoutanyarguments,the
ArangoDBshellwilltrytocontacttheserveronport8529onthelocalhost.FormoreinformationseetheArangoDBShell
documentation.Youmightneedtosetadditionaloptions(endpoint,usernameandpassword)whenconnecting:
unix>./arangosh--server.endpointtcp://127.0.0.1:8529--server.usernameroot
TheshellwillprintitsownversionnumberandifsuccessfullyconnectedtoaservertheversionnumberoftheArangoDBserver.
Command-LineOptions
Use --helptogetalistofcommand-lineoptions:
unix>./arangosh--help
STANDARDoptions:
--audit-log<string>auditlogfiletosavecommandsandresultsto
--configuration<string>readconfigurationfile
--helphelpmessage
--max-upload-size<uint64>maximumsizeofimportchunks(inbytes)(default:500000)
--no-auto-completedisableautocompletion
--no-colorsdeactivatecolorsupport
--pager<string>outputpager(default:"less-X-R-F-L")
--pretty-printprettyprintvalues
--quietnobanner
--temp.path<string>pathfortemporaryfiles(default:"/tmp/arangodb")
--use-pagerusepager
JAVASCRIPToptions:
--javascript.check<string>syntaxcheckcodeJavaScriptcodefromfile
--javascript.execute<string>executeJavaScriptcodefromfile
--javascript.execute-string<string>executeJavaScriptcodefromstring
--javascript.startup-directory<string>startuppathscontainingtheJavaScriptfiles
--javascript.unit-tests<string>donotstartasshell,rununittestsinstead
--jslint<string>donotstartasshell,runjslintinstead
LOGGINGoptions:
--log.level<string>loglevel(default:"info")
CLIENToptions:
--server.connect-timeout<double>connecttimeoutinseconds(default:3)
--server.authentication<bool>whetherornottouseauthentication(default:true)
--server.endpoint<string>endpointtoconnectto,use'none'tostartwithoutaserver(default:"tcp://127.0
.0.1:8529")
--server.password<string>passwordtousewhenconnecting(leaveemptyforprompt)
--server.request-timeout<double>requesttimeoutinseconds(default:300)
--server.username<string>usernametousewhenconnecting(default:"root")
DatabaseWrappers
The dbobjectisavailableinarangoshaswellasonarangodi.e.ifyou'reusingFoxx.Whileitsinterfaceispersistantbetweenthe
arangoshandthearangodimplementations,itsunderpinningisnot.ThearangodimplementationareJavaScriptwrappersaround
ArangoDB'snativeC++implementation,whereasthearangoshimplementationwrapsHTTPaccessestoArangoDB'sRESTfullAPI.
Sowhilethiscodemayproducesimilarresultswhenexecutedinarangoshandarangod,thecpuusageandtimerequiredwillbereally
different:
for(i=0;i<100000;i++){
db.test.save({name:{first:"Jan"},count:i});
}
Sincethearangoshversionwillbedoingaround100kHTTPrequests,andthearangodversionwilldirectlywritetothedatabase.
344
Details
Using arangoshviaunixshebangmechanisms
Inunixoperatingsystemsyoucanstartscriptsbyspecifyingtheinterpreterinthefirstlineofthescript.Thisiscommonlycalled
shebangor hashbang.Youcanalsodothatwith arangosh,i.e.create ~/test.js:
#!/usr/bin/arangosh--javascript.execute
require("internal").print("helloworld")
db._query("FORxINtestRETURNx").toArray()
Notethatthefirstlinehastoendwithablankinordertomakeitwork.M arkitexecutabletotheOS:
#>chmoda+x~/test.js
andfinalytryitout:
#>~/test.js
345
Arangoimp
Arangoimp
ThismanualdescribestheArangoDBimporterarangoimp,whichcanbeusedforbulkimports.
ThemostconvenientmethodtoimportalotofdataintoArangoDBistousethearangoimpcommand-linetool.Itallowsyoutoimport
datarecordsfromafileintoanexistingdatabasecollection.
Itispossibletoimportdocumentkeyswiththedocumentsusingthe_keyattribute.Whenimportingintoanedgecollection,itis
mandatorythatallimporteddocumentshavethe_fromand_toattributes,andthattheycontainvalidreferences.
Let'sassumeforthefollowingexamplesyouwanttoimportuserdataintoanexistingcollectionnamed"users"ontheserver.
ImportingDataintoanArangoDBDatabase
ImportingJSON-encodedData
Let'sfurtherassumetheimportathandisencodedinJSON.We'llbeusingtheseexampleuserrecordstoimport:
{"name":{"first":"John","last":"Connor"},"active":true,"age":25,"likes":["swimming"]}
{"name":{"first":"Jim","last":"O'Brady"},"age":19,"likes":["hiking","singing"]}
{"name":{"first":"Lisa","last":"Jones"},"dob":"1981-04-09","likes":["running"]}
Toimporttheserecords,allyouneedtodoistoputthemintoafile(withonelineforeachrecordtoimport)andrunthefollowing
command:
>arangoimp--file"data.json"--typejsonl--collection"users"
Thiswilltransferthedatatotheserver,importtherecords,andprintastatussummary.Toshowtheintermediateprogressduringthe
importprocess,theoption--progresscanbeadded.Thisoptionwillshowthepercentageoftheinputfilethathasbeensenttothe
server.Thiswillonlybeusefulforbigimportfiles.
>arangoimp--file"data.json"--typejson--collectionusers--progresstrue
Itisalsopossibletousetheoutputofanothercommandasaninputforarangoimp.Forexample,thefollowingshellcommandcanbe
usedtopipedatafromthe catprocesstoarangoimp:
>catdata.json|arangoimp--file---typejson--collectionusers
Notethatyouhavetouse --file-ifyouwanttouseanothercommandasinputforarangoimp.Noprogresscanbereportedforsuch
importsasthesizeoftheinputwillbeunknowntoarangoimp.
Bydefault,theendpointtcp://127.0.0.1:8529willbeused.Ifyouwanttospecifyadifferentendpoint,youcanusethe--server.endpoint
option.Youprobablywanttospecifyadatabaseuserandpasswordaswell.Youcandosobyusingtheoptions--server.usernameand-server.password.Ifyoudonotspecifyapassword,youwillbepromptedforone.
>arangoimp--server.endpointtcp://127.0.0.1:8529--server.usernameroot--file"data.json"--typejson--collection"users"
Notethatthecollection(usersinthiscase)mustalreadyexistortheimportwillfail.Ifyouwanttocreateanewcollectionwiththe
importdata,youneedtospecifythe--create-collectionoption.Notethatbydefaultitwillcreateadocumentcollectionandnoede
collection.
>arangoimp--file"data.json"--typejson--collection"users"--create-collectiontrue
Tocreateanedgecollectioninstead,usethe--create-collection-typeoptionandsetittoedge:
>arangoimp--file"data.json"--collection"myedges"--create-collectiontrue--create-collection-typeedge
346
Arangoimp
Whenimportingdataintoanexistingcollectionitisoftenconvenienttofirstremovealldatafromthecollectionandthenstarttheimport.
Thiscanbeachievedbypassingthe--overwriteparametertoarangoimp.Ifitissettotrue,anyexistingdatainthecollectionwillbe
removedpriortotheimport.Notethatanyexistingindexdefinitionsforthecollectionwillbepreservedevenif--overwriteissettotrue.
>arangoimp--file"data.json"--typejson--collection"users"--overwritetrue
AstheimportfilealreadycontainsthedatainJSONformat,attributenamesanddatatypesarefullypreserved.Ascanbeseeninthe
exampledata,thereisnoneedforalldatarecordstohavethesameattributenamesortypes.Recordscanbeinhomogeneous.
Pleasenotethatbydefault,arangoimpwillimportdataintothespecifiedcollectioninthedefaultdatabase(_system).Tospecifya
differentdatabase,usethe--server.databaseoptionwheninvokingarangoimp.
Thetoolalsosupportsparallelimports,withmultiplethreads.Usingmultiplethreadsmayprovideaspeedup,especiallywhenusingthe
RocksDBstorageengine.Tospecifythenumberofparallelthreadsusethe --threadsoption:
>arangoimp--threads4--file"data.json"--typejson--collection"users"
Notethatusingmultiplethreadsmayleadtoanon-sequentialimportoftheinputdata.Datathatappearslaterintheinputfilemaybe
importedearlierthandatathatappearsearlierintheinputfile.Thisisnormallynotaproblembutmaycauseissueswhenwhenthereare
datadependenciesorduplicatesintheimportdata.Inthiscase,thenumberofthreadsshouldbesetto1.
JSONinputfileformats
Note:arangoimpsupportstwoformatswhenimportingJSONdatafromafile.Thefirstformatthatwealsousedaboveiscommonly
knownasjsonl).However,incontrasttotheJSONLspecificationitrequirestheinputfiletocontainonecompleteJSONdocumentin
eachline,e.g.
{"_key":"one","value":1}
{"_key":"two","value":2}
{"_key":"foo","value":"bar"}
...
SoonecouldarguethatthisisonlyasubsetofJSONL.
Theaboveformatcanbeimportedsequentiallybyarangoimp.Itwillreaddatafromtheinputfileinchunksandsenditinbatchestothe
server.Eachbatchwillbeaboutasbigasspecifiedinthecommand-lineparameter--batch-size.
AnalternativeistoputonebigJSONdocumentintotheinputfilelikethis:
[
{"_key":"one","value":1},
{"_key":"two","value":2},
{"_key":"foo","value":"bar"},
...
]
Thisformatallowslinebreakswithintheinputfileasrequired.Thedownsideisthatthewholeinputfilewillneedtobereadby
arangoimpbeforeitcansendthefirstbatch.Thismightbeaproblemiftheinputfileisbig.Bydefault,arangoimpwillallowimporting
suchfilesuptoasizeofabout16M B.
Ifyouwanttoallowyourarangoimpinstancetousemorememory,youmaywanttoincreasethemaximumfilesizebyspecifyingthe
command-lineoption--batch-size.Forexample,tosetthebatchsizeto32M B,usethefollowingcommand:
>arangoimp--file"data.json"--typejson--collection"users"--batch-size33554432
Pleasealsonotethatyoumayneedtoincreasethevalueof--batch-sizeifasingledocumentinsidetheinputfileisbiggerthanthevalueof
--batch-size.
ImportingCSVData
347
Arangoimp
arangoimpalsooffersthepossibilitytoimportdatafromCSVfiles.ThiscomeshandywhenthedataathandisinCSVformatalready
andyoudon'twanttospendtimeconvertingthemtoJSONfortheimport.
ToimportdatafromaCSVfile,makesureyourfilecontainstheattributenamesinthefirstrow.Allthefollowinglinesinthefilewillbe
interpretedasdatarecordsandwillbeimported.
TheCSVimportrequiresthedatatohaveahomogeneousstructure.Allrecordsmusthaveexactlythesameamountofcolumnsasthere
areheaders.
Thecellvaluescanhavedifferentdatatypesthough.Ifacelldoesnothaveanyvalue,itcanbeleftemptyinthefile.Thesevalueswill
notbeimportedsotheattributeswillnot"bethere"indocumentcreated.Valuesenclosedinquoteswillbeimportedasstrings,soto
importnumericvalues,booleanvaluesorthenullvalue,don'tenclosethevalueinquotesinyourfile.
We'llbeusingthefollowingimportfortheCSVimport:
"first","last","age","active","dob"
"John","Connor",25,true,
"Jim","O'Brady",19,,
"Lisa","Jones",,,"1981-04-09"
Hans,dosSantos,0123,,
Wayne,Brewer,,false,
Thecommandlinetoexecutetheimportis:
>arangoimp--file"data.csv"--typecsv--collection"users"
Theabovedatawillbeimportedinto5documentswhichwilllookasfollows:
{"first":"John","last":"Connor","active":true,"age":25}
{"first":"Jim","last":"O'Brady","age":19}
{"first":"Lisa","last":"Jones","dob":"1981-04-09"}
{"first":"Hans","last":"dosSantos","age":123}
{"first":"Wayne","last":"Brewer","active":false}
Ascanbeseen,valuesleftcompletelyemptyintheinputfilewillbetreatedasabsent.Numericvaluesnotenclosedinquoteswillbe
treatedasnumbers.Notethatleadingzerosinnumericvalueswillberemoved.Toimportnumberswithleadingzeros,pleaseusestrings.
Theliteralstrueandfalsewillbetreatedasbooleansiftheyarenotenclosedinquotes.Othervaluesnotenclosedinquoteswillbetreated
asstrings.Anyvaluesenclosedinquoteswillbetreatedasstrings,too.
Stringvaluescontainingthequotecharacterortheseparatormustbeenclosedwithquotecharacters.Withinastring,thequotecharacter
itselfmustbeescapedwithanotherquotecharacter(orwithabackslashifthe--backslash-escapeoptionisused).
Notethatthequoteandseparatorcharacterscanbeadjustedviathe--quoteand--separatorargumentswheninvokingarangoimp.The
quotecharacterdefaultstothedoublequote(").Tousealiteralquoteinastring,youcanusetwoquotecharacters.Tousebackslashfor
escapingquotecharacters,pleasesettheoption--backslash-escapetotrue.
TheimportersupportsWindows(CRLF)andUnix(LF)linebreaks.Linebreaksmightalsooccurinsidevaluesthatareenclosedwiththe
quotecharacter.
Here'sanexampleforusingliteralquotesandnewlinesinsidevalues:
"name","password"
"Foo","r4ndom""123!"
"Bar","wow!
thisisa
multinepassword!"
"Bartholomew""Bart""Simpson","Milhouse"
Extrawhitespaceattheendofeachlinewillbeignored.Whitespaceatthestartoflinesorbetweenfieldvalueswillnotbeignored,so
pleasemakesurethatthereisnoextrawhitespaceinfrontofvaluesorbetweenthem.
ImportingTSVData
348
Arangoimp
Youmayalsoimporttab-separatedvalues(TSV)fromafile.Thisformatisverysimple:everylineinthefilerepresentsadatarecord.
Thereisnoquotingorescaping.Thatalsomeansthattheseparatorcharacter(whichdefaultstothetabstopsymbol)mustnotbeused
anywhereintheactualdata.
AswithCSV,thefirstlineintheTSVfilemustcontaintheattributenames,andalllinesmusthaveanidenticalnumberofvalues.
Ifadifferentseparatorcharacterorstringshouldbeused,itcanbespecifiedwiththe--separatorargument.
AnexamplecommandlinetoexecutetheTSVimportis:
>arangoimp--file"data.tsv"--typetsv--collection"users"
AttributeNameTranslation
FortheCSVandTSVinputformats,attributenamescanbetranslatedautomatically.Thisisusefulincasetheimportfilehasdifferent
attributenamesthanthosethatshouldbeusedinArangoDB.
Acommonusecaseistorenamean"id"columnfromtheinputfileinto"_key"asitisexpectedbyArangoDB.Todothis,specifythe
followingtranslationwheninvokingarangoimp:
>arangoimp--file"data.csv"--typecsv--translate"id=_key"
Othercommoncasesaretorenamecolumnsintheinputfileto_fromand_to:
>arangoimp--file"data.csv"--typecsv--translate"from=_from"--translate"to=_to"
Thetranslateoptioncanbespecifiedmultipletypes.Thesourceattributenameandthetargetattributemustbeseparatedwitha=.
ImportingintoanEdgeCollection
arangoimpcanalsobeusedtoimportdataintoanexistingedgecollection.Theimportdatamust,foreachedgetoimport,containatleast
the_fromand_toattributes.Theseindicatewhichothertwodocumentstheedgeshouldconnect.Itisnecessarythattheseattributesare
setforallrecords,andpointtovaliddocumentidsinexistingcollections.
Examples
{"_from":"users/1234","_to":"users/4321","desc":"1234isconnectedto4321"}
Note:Theedgecollectionmustalreadyexistwhentheimportisstarted.Usingthe--create-collectionflagwillnotworkbecause
arangoimpwillalwaystrytocreatearegulardocumentcollectionifthetargetcollectiondoesnotexist.
Updatingexistingdocuments
Bydefault,arangoimpwilltrytoinsertalldocumentsfromtheimportfileintothespecifiedcollection.Incasetheimportfilecontains
documentsthatarealreadypresentinthetargetcollection(matchingisdoneviathe_keyattributes),thenadefaultarangoimprunwillnot
importthesedocumentsandcomplainaboutuniquekeyconstraintviolations.
However,arangoimpcanbeusedtoupdateorreplaceexistingdocumentsincasetheyalreadyexistinthetargetcollection.Itprovidesthe
command-lineoption--on-duplicatetocontrolthebehaviorincaseadocumentisalreadypresentinthedatabase.
Thedefaultvalueof--on-duplicateiserror.Thismeansthatwhentheimportfilecontainsadocumentthatispresentinthetarget
collectionalready,thentryingtore-insertadocumentwiththesame_keyvalueisconsideredanerror,andthedocumentinthedatabase
willnotbemodified.
Otherpossiblevaluesfor--on-duplicateare:
update:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbeupdatedbyarangoimp.
updatewillperformapartialupdateoftheexistingdocument,modifyingonlytheattributesthatarepresentintheimportfileand
leavingallotherattributesuntouched.
Thevaluesofsystemattributes_id,_key,_rev,_fromand_tocannotbeupdatedorreplacedinexistingdocuments.
349
Arangoimp
replace:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbereplacebyarangoimp.
replacewillreplacetheexistingdocumententirely,resultinginadocumentwithonlytheattributesspecifiedintheimportfile.
Thevaluesofsystemattributes_id,_key,_rev,_fromand_tocannotbeupdatedorreplacedinexistingdocuments.
ignore:eachdocumentpresentintheimportfilethatisalsopresentinthetargetcollectionalreadywillbeignoredandnotmodified
inthetargetcollection.
When--on-duplicateissettoeitherupdateorreplace,arangoimpwillreturnthenumberofdocumentsupdated/replacedintheupdated
returnvalue.Whensettoanothervalue,thevalueofupdatedwillalwaysbezero.When--on-duplicateissettoignore,arangoimpwill
returnthenumberofignoreddocumentsintheignoredreturnvalue.Whensettoanothervalue,ignoredwillalwaysbezero.
Itispossibletoperformacombinationofinsertsandupdates/replaceswithasinglearangoimprun.When--on-duplicateissettoupdate
orreplace,alldocumentspresentintheimportfilewillbeinsertedintothetargetcollectionprovidedtheyarevalidanddonotalready
existwiththespecified_key.Documentsthatarealreadypresentinthetargetcollection(identifiedby_keyattribute)willinsteadbe
updated/replaced.
Arangoimpresultoutput
Anarangoimpimportrunwillprintoutthefinalresultsonthecommandline.Itwillshowthe
numberofdocumentscreated(created)
numberofdocumentsupdated/replaced(updated/replaced,onlynon-zeroif--on-duplicatewassettoupdateorreplace,seebelow)
numberofwarningsorerrorsthatoccurredontheserverside(warnings/errors)
numberofignoreddocuments(onlynon-zeroif--on-duplicatewassettoignore).
Example
created:2
warnings/errors:0
updated/replaced:0
ignored:0
ForCSVandTSVimports,thetotalnumberofinputfilelinesreadwillalsobeprinted(linesread).
arangoimpwillalsoprintoutdetailsaboutwarningsanderrorsthathappenedontheserver-side(ifany).
AttributeNamingandSpecialAttributes
AttributeswhosenamesstartwithanunderscorearetreatedinaspecialwaybyArangoDB:
theoptional_keyattributecontainsthedocument'skey.Ifspecified,thevaluemustbeformallyvalid(e.g.mustbeastringand
conformtothenamingconventions).Additionally,thekeyvaluemustbeuniquewithinthecollectiontheimportisrunfor.
_from:whenimportingintoanedgecollection,thisattributecontainstheidofoneofthedocumentsconnectedbytheedge.The
valueof_frommustbeasyntacticallyvaliddocumentidandthereferredcollectionmustexist.
_to:whenimportingintoanedgecollection,thisattributecontainstheidoftheotherdocumentconnectedbytheedge.Thevalueof
_tomustbeasyntacticallyvaliddocumentidandthereferredcollectionmustexist.
_rev:thisattributecontainstherevisionnumberofadocument.However,therevisionnumbersaremanagedbyArangoDBand
cannotbespecifiedonimport.Thusanyvalueinthisattributeisignoredonimport.
Ifyouimportvaluesinto_key,youshouldmakesuretheyarevalidandunique.
Whenimportingdataintoanedgecollection,youshouldmakesurethatallimportdocumentscan_fromand_toandthattheirvalues
pointtoexistingdocuments.
Toavoidspecifyingcompletedocumentids(consistingofcollectionnamesanddocumentkeys)for_fromand_tovalues,therearethe
options--from-collection-prefixand--to-collection-prefix.Ifspecified,thesevalueswillbeautomaticallyprependedtoeachvaluein
_from(or_toresp.).Thisallowsspecifyingonlydocumentkeysinside_fromand/or_to.
Example
>arangoimp--from-collection-prefixusers--to-collection-prefixproducts...
350
Arangoimp
Importingthefollowingdocumentwillthencreateanedgebetweenusers/1234andproducts/4321:
{"_from":"1234","_to":"4321","desc":"users/1234isconnectedtoproducts/4321"}
351
Arangodump
DumpingDatafromanArangoDBdatabase
TodumpdatafromanArangoDBserverinstance,youwillneedtoinvokearangodump.Dumpscanbere-importedwitharangorestore.
arangodumpcanbeinvokedbyexecutingthefollowingcommand:
unix>arangodump--output-directory"dump"
ThiswillconnecttoanArangoDBserveranddumpallnon-systemcollectionsfromthedefaultdatabase(_system)intoanoutput
directorynameddump.Invokingarangodumpwillfailiftheoutputdirectoryalreadyexists.Thisisanintentionalsecuritymeasureto
preventyoufromaccidentallyoverwritingalreadydumpeddata.Ifyouarepositivethatyouwanttooverwritedataintheoutput
directory,youcanusetheparameter--overwritetruetoconfirmthis:
unix>arangodump--output-directory"dump"--overwritetrue
arangodumpwillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabaseor
adifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:
--server.database:nameofthedatabasetoconnectto
--server.endpoint:endpointtoconnectto
--server.username:username
--server.password:passwordtouse(omitthisandyou'llbepromptedforthepassword)
--server.authentication:whetherornottouseauthentication
Here'sanexampleofdumpingdatafromanon-standardendpoint,usingadedicateddatabasename:
unix>arangodump--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--output-directory
"dump"
Whenfinished,arangodumpwillprintoutasummarylinewithsomeaggregatestatisticsaboutwhatitdid,e.g.:
Processed43collection(s),wrote408173500byte(s)intodatafiles,sent88batch(es)
Bydefault,arangodumpwilldumpbothstructuralinformationanddocumentsfromallnon-systemcollections.Toadjustthis,thereare
thefollowingcommand-linearguments:
--dump-data:settotruetoincludedocumentsinthedump.Settofalsetoexcludedocuments.Thedefaultvalueistrue.
--include-system-collections:whetherornottoincludesystemcollectionsinthedump.Thedefaultvalueisfalse.
Forexample,toonlydumpstructuralinformationofallcollections(includingsystemcollections),use:
unix>arangodump--dump-datafalse--include-system-collectionstrue--output-directory"dump"
Torestrictthedumptojustspecificcollections,thereisisthe--collectionoption.Itcanbespecifiedmultipletimesifrequired:
unix>arangodump--collectionmyusers--collectionmyvalues--output-directory"dump"
Structuralinformationforacollectionwillbesavedinfileswithnamepattern.structure.json.EachstructurefilewillcontainsaJSON
objectwiththeseattributes:
parameters:containsthecollectionproperties
indexes:containsthecollectionindexes
Documentdataforacollectionwillbesavedinfileswithnamepattern.data.json.Eachlineinadatafileisadocumentinsertion/update
ordeletionmarker,alongsidewithsomemetadata.
StartingwithVersion2.1ofArangoDB,thearangodumptoolalsosupportssharding.Simplypointittooneofthecoordinatorsandit
willbehaveexactlyasdescribedabove,workingonshardedcollectionsinthecluster.
352
Arangodump
However,asopposedtothesingleinstancesituation,thisoperationdoesnotguaranteetodumpaconsistentsnapshotifwrite
operationshappenduringthedumpoperation.Itisthereforerecommendednottoperformanydata-modifcationoperationsonthe
clusterwhilstarangodumpisrunning.
Asabove,theoutputwillbeonestructuredescriptionfileandonedatafilepershardedcollection.Notethatthedatainthedatafileis
sortedfirstbyshardsandwithineachshardbyascendingtimestamp.Thestructuralinformationofthecollectioncontainsthenumberof
shardsandtheshardkeys.
NotethattheversionofthearangodumpclienttoolneedstomatchtheversionoftheArangoDBserveritconnectsto.Bydefault,
arangodumpwillproduceadumpthatcanberestoredwiththearangorestoretoolofthesameversion.Anexceptionisarangodumpin
3.0,whichsupportsdumpingdatainaformatcompatiblewithArangoDB2.8.Inordertoproducea2.8-compatibledumpwitha3.0
ArangoDB,pleasespecifytheoption --compat28truewheninvokingarangodump.
unix>arangodump--compat28true--collectionmyvalues--output-directory"dump"
Advancedclusteroptions
Startingwithversion3.1.17,collectionsmaybecreatedwithsharddistributionidenticaltoanexistingprototypicalcollection;i.e.shards
aredistributedintheverysamepatternasintheprototypecollection.Suchcollectionscannotbedumpedwithoutthereference
collectionorarangodumpwithyieldanerror.
unix>arangodump--collectionclonedCollection--output-directory"dump"
ERRORCollectionclonedCollection'ssharddistributionisbasedonathatofcollectionprototypeCollection,whichisnotdumpe
dalong.Youmaydumpthecollectionregardlessofthemissingprototypecollectionbyusingthe--ignore-distribute-shards-lik
e-errorsparameter.
Therearetwowaystoapproachthatproblem:Solveit,i.e.dumptheprototypecollectionalong:
unix>arangodump--collectionclonedCollection--collectionprototypeCollection--output-directory"dump"
Processed2collection(s),wrote81920byte(s)intodatafiles,sent1batch(es)
Oroverridethatbehaviourtobeabletodumpthecollectionindividually.
unix>arangodump--collectionBclonedCollection--output-directory"dump"--ignore-distribute-shards-like-errors
Processed1collection(s),wrote34217byte(s)intodatafiles,sent1batch(es)
Nothatinconsequence,restoringsuchacollectionwithoutitsprototypeisaffected.arangorestore
353
Arangorestore
Arangorestore
Toreloaddatafromadumppreviouslycreatedwitharangodump,ArangoDBprovidesthearangorestoretool.
ReloadingDataintoanArangoDBdatabase
Invokingarangorestore
arangorestorecanbeinvokedfromthecommand-lineasfollows:
unix>arangorestore--input-directory"dump"
ThiswillconnecttoanArangoDBserverandreloadstructuralinformationanddocumentsfoundintheinputdirectorydump.Pleasenote
thattheinputdirectorymusthavebeencreatedbyrunningarangodumpbefore.
arangorestorewillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabase
oradifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:
--server.database:nameofthedatabasetoconnectto
--server.endpoint:endpointtoconnectto
--server.username:username
--server.password:passwordtouse(omitthisandyou'llbepromptedforthepassword)
--server.authentication:whetherornottouseauthentication
Sinceversion2.6arangorestoreprovidestheoption--create-database.Settingthisoptiontotruewillcreatethetargetdatabaseifitdoes
notexist.Whencreatingthetargetdatabase,theusernameandpasswordspassedtoarangorestore(inoptions--server.usernameand-server.password)willbeusedtocreateaninitialuserforthenewdatabase.
Here'sanexampleofreloadingdatatoanon-standardendpoint,usingadedicateddatabasename:
unix>arangorestore--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--input-directo
ry"dump"
Tocreatethetargetdatabasewherestoring,useacommandlikethis:
unix>arangorestore--server.usernamebackup--server.databasenewdb--create-databasetrue--input-directory"dump"
arangorestorewillprintoutitsprogresswhilerunning,andwillendwithalineshowingsomeaggregatestatistics:
Processed2collection(s),read2256byte(s)fromdatafiles,sent2batch(es)
Bydefault,arangorestorewillre-createallnon-systemcollectionsfoundintheinputdirectoryandloaddataintothem.Ifthetarget
databasealreadycontainscollectionswhicharealsopresentintheinputdirectory,theexistingcollectionsinthedatabasewillbedropped
andre-createdwiththedatafoundintheinputdirectory.
Thefollowingparametersareavailabletoadjustthisbehavior:
--create-collection:settotruetocreatecollectionsinthetargetdatabase.Ifthetargetdatabasealreadycontainsacollectionwiththe
samename,itwillbedroppedfirstandthenre-createdwiththepropertiesfoundintheinputdirectory.Settofalsetokeepexisting
collectionsinthetargetdatabase.Ifsettofalseandarangorestoreencountersacollectionthatispresentinboththetargetdatabase
andtheinputdirectory,itwillabort.Thedefaultvalueistrue.
--import-data:settotruetoloaddocumentdataintothecollectionsinthetargetdatabase.Settofalsetonotloadanydocument
data.Thedefaultvalueistrue.
--include-system-collections:whetherornottoincludesystemcollectionswhenre-creatingcollectionsorreloadingdata.Thedefault
valueisfalse.
Forexample,to(re-)createallnon-systemcollectionsandloaddocumentdataintothem,use:
354
Arangorestore
unix>arangorestore--create-collectiontrue--import-datatrue--input-directory"dump"
Thiswilldroppotentiallyexistingcollectionsinthetargetdatabasethatarealsopresentintheinputdirectory.
Toincludesystemcollectionstoo,use--include-system-collectionstrue:
unix>arangorestore--create-collectiontrue--import-datatrue--include-system-collectionstrue--input-directory"dump"
To(re-)createallnon-systemcollectionswithoutloadingdocumentdata,use:
unix>arangorestore--create-collectiontrue--import-datafalse--input-directory"dump"
Thiswillalsodropexistingcollectionsinthetargetdatabasethatarealsopresentintheinputdirectory.
Tojustloaddocumentdataintoallnon-systemcollections,use:
unix>arangorestore--create-collectionfalse--import-datatrue--input-directory"dump"
Torestrictreloadingtojustspecificcollections,thereisisthe--collectionoption.Itcanbespecifiedmultipletimesifrequired:
unix>arangorestore--collectionmyusers--collectionmyvalues--input-directory"dump"
Collectionswillbeprocessedbyinalphabeticalorderbyarangorestore,withalldocumentcollectionsbeingprocessedbeforealledge
collections.Thisistoensurethatreloadingdataintoedgecollectionswillhavethedocumentcollectionslinkedinedges(_fromand_to
attributes)loaded.
RestoringRevisionIdsandCollectionIds
arangorestorewillreloaddocumentandedgesdatawiththeexactsame_key,_fromand_tovaluesfoundintheinputdirectory.
However,whenloadingdocumentdata,itwillassignitsownvaluesforthe_revattributeofthereloadeddocuments.Thoughthis
differenceisintentional(normally,everyservershouldcreateitsown_revvalues)theremightbesituationswhenitisrequiredtore-use
theexactsame_revvaluesforthereloadeddata.Thiscanbeachievedbysettingthe--recycle-idsparametertotrue:
unix>arangorestore--collectionmyusers--collectionmyvalues--recycle-idstrue--input-directory"dump"
Notethatsetting--recycle-idstotruewillalsocausecollectionstobe(re-)createdinthetargetdatabasewiththeexactsamecollectionid
asintheinputdirectory.Anypotentiallyexistingcollectioninthetargetdatabasewiththesamecollectionidwillthenbedropped.
Setting--recycle-idstofalseoromittingitwillonlyusethecollectionnamefromtheinputdirectoryandallowthetargetdatabaseto
createthecollectionwithadifferentid(thoughwiththesamename)thanintheinputdirectory.
ReloadingDataintoadifferentCollection
Withsomecreativityyoucanusearangodumpandarangorestoretotransferdatafromonecollectionintoanother(eitheronthesame
serverornot).Forexample,tocopydatafromacollectionmyvaluesindatabasemydbintoacollectionmycopyvaluesindatabasemycopy,
youcanstartwiththefollowingcommand:
unix>arangodump--collectionmyvalues--server.databasemydb--output-directory"dump"
Thiswillcreatetwofiles,myvalues.structure.jsonandmyvalues.data.json,intheoutputdirectory.Toloaddatafromthedatafileintoan
existingcollectionmycopyvaluesindatabasemycopy,renamethefilestomycopyvalues.structure.jsonandmycopyvalues.data.json.After
that,runthefollowingcommand:
unix>arangorestore--collectionmycopyvalues--server.databasemycopy--input-directory"dump"
Usingarangorestorewithsharding
355
Arangorestore
AsofVersion2.1thearangorestoretoolsupportssharding.Simplypointittooneofthecoordinatorsinyourclusteranditwillworkas
usualbutonshardedcollectionsinthecluster.
Ifarangorestoreisaskedtodropandre-createacollection,itwillusethesamenumberofshardsandthesameshardkeysaswhenthe
collectionwasdumped.Thedistributionoftheshardstotheserverswillalsobethesameasatthetimeofthedump.Thismeansin
particularthatDBserverswiththesameIDsasbeforemustbepresentintheclusterattimeoftherestore.
Ifacollectionwasdumpedfromasingleinstance,onecanmanuallyaddthestructuraldescriptionfortheshardkeysandthenumberand
distributionoftheshardsandthentherestoreintoaclusterwillwork.
IfyourestoreacollectionthatwasdumpedfromaclusterintoasingleArangoDBinstance,thenumberofshardsandtheshardkeyswill
silentlybeignored.
Notethatinacluster,everynewlycreatedcollectionwillhaveanewID,itisnotpossibletoreusetheIDfromtheoriginallydumped
collection.ThisisforsafetyreasonstoensureconsistencyofIDs.
Restoringcollectionswithshardingprototypes
arangorestorewillyieldanerror,whiletryingtorestoreacollection,whosesharddistributionfollowsacollection,whichdoesnotexist
intheclusterandwhichwasnotdumpedalong:
unix>arangorestore--collectionclonedCollection--server.databasemydb--input-directory"dump"
ERRORgoterrorfromserver:HTTP500(InternalServerError):ArangoError1486:mustnothaveadistributeShardsLikeattribute
pointingtoanunknowncollection
Processed0collection(s),read0byte(s)fromdatafiles,sent0batch(es)
Thecollectioncanberestoredbyoverridingtheerrormessageasfollows:
unix>arangorestore--collectionclonedCollection--server.databasemydb--input-directory"dump"--ignore-distribute-shards-li
ke-errors
356
Arangoexport
ExportingDatafromanArangoDBdatabase
ToexportdatafromanArangoDBserverinstance,youwillneedtoinvokearangoexport.arangoexportcanbeinvokedbyexecutingthe
followingcommand:
unix>arangoexport--collectiontest--output-directory"dump"
Thisexportsthecollectionstestintothedirectorydumpasonebigjsonarray.Everyentryinthisarrayisonedocumentfromthe
collectionwithoutaspecificorder.Toexportmorethanonecollectionatatimespecifymultiple--collectionoptions.
Thedefaultoutputdirectoryisexport.
arangoexportwillbydefaultconnecttothe_systemdatabaseusingthedefaultendpoint.Ifyouwanttoconnecttoadifferentdatabaseor
adifferentendpoint,oruseauthentication,youcanusethefollowingcommand-lineoptions:
--server.database:nameofthedatabasetoconnectto
--server.endpoint:endpointtoconnectto
--server.username:username
--server.password:passwordtouse(omitthisandyou'llbepromptedforthepassword)
--server.authentication:whetherornottouseauthentication
Here'sanexampleofexportingdatafromanon-standardendpoint,usingadedicateddatabasename:
unix>arangoexport--server.endpointtcp://192.168.173.13:8531--server.usernamebackup--server.databasemydb--collectiontes
t--output-directory"my-export"
Whenfinished,arangoexportwillprintoutasummarylinewithsomeaggregatestatisticsaboutwhatitdid,e.g.:
Processed2collection(s),wrote9031763Byte(s),78HTTPrequest(s)
ExportJSON
unix>arangoexport--typejson--collectiontest
Thisexportsthecollectiontestintotheoutputdirectoryexportasonejsonarray.Everyarrayentryisonedocumentfromthecollection
test
ExportJSONL
unix>arangoexport--typejsonl--collectiontest
Thisexportsthecollectiontestintotheoutputdirectoryexportasjsonl.Everylineintheexportisonedocumentfromthecollectiontest
asjson.
ExportCSV
unix>arangoexport--typecsv--collectiontest--fields_key,_id,_rev
ThisexportsthecollectiontestintotheoutputdirectoryexportasCSV.Thefirstlinecontainstheheaderwithallfieldnames.Eachlineis
onedocumentrepresentedasCSVandseparatedwithacomma.ObjectsandArraysarerepresentedasaJSONstring.
357
Arangoexport
ExportXML
unix>arangoexport--typexml--collectiontest
ThisexportsthecollectiontestintotheoutputdirectoryexportasgenericXM L.TherootelementofthegeneratedXM Lfileisnamed
collection.EachdocumentinthecollectionisexportedinadocXM Lattribute.Eachdocumentattributeisexportinagenericattelement,
whichhasatypeattributeindicatingtheattributevalue,andavalueattributecontainingtheattribute'svalue.
ExportXGMML
XGM M LisanXM LapplicationbasedonGM L.ToviewtheXGM M LfileyoucanuseforexampleCytoscape.
importantnote
Ifyouexportallattributes(--xgmml-label-onlyfalse)keepinmindthataatrributenamestypehavetobethesametypeforall
documents.Itwontworkifyouhaveaattributenamedrankthatisinonedocumentastringandinanotherdocumentainteger.
Bad
//doc1
{
"rank":1
}
//doc2
{
"rank":"2"
}
Good
//doc1
{
"rank":1
}
//doc2
{
"rank":2
}
XGMMLspecificoptions
--xgmml-label-attributespecifythenameoftheattributethatwillbecomethelabelinthexgmmlfile.
--xgmml-label-onlysettotruewillonlyexportthelabelwithoutanyattributesinedgesornodes.
exportbasedoncollections
unix>arangoexport--typexgmml--graph-namemygraph--collectionvertex--collectionedge
Thisexportstheaunnamedgraphwithvertexcollectionvertexandedgecollectionedgeintothexgmmlfilemygraph.xgmml.
exportbasedonanamedgraph
unix>arangoexport--typexgmml--graph-namemygraph
358
Arangoexport
Thisexportsthenamedgraphmygraphintothexgmmlfilemygraph.xgmml.
exportXGMMLwithoutattributes
unix>arangoexport--typexgmml--graph-namemygraph--xgmml-label-onlytrue
Thisexportsthenamedgraphmygraphintothexgmmlfilemygraph.xgmmlwithoutthe<att>taginnodesandedges.
exportXGMMLwithaspecificlabel
unix>arangoexport--typexgmml--graph-namemygraph--xgmml-label-attributename
Thisexportsthenamedgraphmygraphintothexgmmlfilemygraph.xgmmlwithalabelfromdocumentsattributenameinsteadofthe
defaultattributelabel.
ExportviaAQLquery
unix>arangoexport--typejsonl--query"forbookinbooksfilterbook.sells>100returnbook"
Exportviaanaqlqueryallowsyoutoexportthereturneddataasthetypespecifiedwith--type.Theexampleexportsallbooksasjsonl
thataresoldmorethan100times.
359
M anagingUsers
ManagingUsers
TheusermanagementinArangoDB3issimilartotheonesfoundinM ySQL,PostgreSQL,orotherdatabasesystems.
Usermanagementispossibleinthewebinterfaceandinarangoshwhileloggedontothe_systemdatabase.
ActionsandAccessLevels
AnArangoDBservercontainsalistofusers.Italsodefinesvariousaccesslevelsthatcanbeassignedtoauser(fordetails,seebelow)and
thatareneededtoperformcertainactions.Theseactionscanbegroupedintothreecategories:
serveractions
databaseactions
collectionactions
Theserveractionsare
createuser:allowstocreateanewuser.
updateuser:allowstochangetheaccesslevelsanddetailsofanexistinguser.
dropuser:allowstodeleteanexistinguser.
createdatabase:allowstocreateanewdatabase.
dropdatabase:allowstodeleteanexistingdatabase.
Thedatabaseactionsaretiedtoagivendatabase,andaccesslevelsmustbesetforeachdatabaseindividually.Foragivendatabasethe
actionsare
createcollection:allowstocreateanewcollectioninthegivendatabase.
updatecollection:allowstoupdatepropertiesofanexistingcollection.
dropcollection:allowstodeleteanexistingcollection.
createindex:allowstocreateanindexforanexistingcollectioninthegivendatabase.
dropindex:allowstodeleteanindexofanexistingcollectioninthegivendatabase.
Thecollectionactionsaretiedtoagivencollectionofagivendatabase,andaccesslevelsmustbesetforeachcollectionindividually.For
agivencollectiontheactionsare
readdocument:readadocumentofthegivencollection.
createdocument:createsanewdocumentinthegivencollection.
modifydocument:modifiesanexistingdocumentofthegivencollection,thiscanbeanupdateorreplaceoperation.
dropdocument:deletesanexistingdocumentofthegivencollection.
truncatecollection:deletesalldocumentsofagivencollection.
Toperformactionsontheserverleveltheuserneedsatleastthefollowingaccesslevels.TheaccesslevelsareAdministrateandNo
access:
serveraction
serverlevel
createadatabase
Administrate
dropadatabase
Administrate
createauser
Administrate
updateauser
Administrate
updateuseraccesslevel
Administrate
360
M anagingUsers
dropauser
Administrate
Toperformactionsinaspecificdatabase(likecreatingordroppingcollections),auserneedsatleastthefollowingaccesslevel.The
possibleaccesslevelsfordatabasesareAdministrate,AccessandNoaccess.TheaccesslevelsforcollectionsareRead/Write,ReadOnly
andNoAccess.
databaseaction
databaselevel
collectionlevel
createcollection
Administrate
Read/Write
listcollections
Access
ReadOnly
renamecollection
Administrate
Read/Write
modifycollectionproperties
Administrate
Read/Write
readproperties
Access
ReadOnly
dropcollection
Administrate
Read/Write
createanindex
Administrate
Read/Write
dropanindex
Administrate
Read/Write
seeindexdefinition
Access
ReadOnly
NotethattheaccesslevelAccessforadatabaseisalwaysrequiredtoperformanyactiononacollectioninthatdatabase.
Forcollectionsauserneedsthefollowingaccesslevelstothegivendatabaseandthegivencollection.Theaccesslevelsforthedatabase
areAdministrate,AccessandNoaccess.TheaccesslevelsforthecollectionareRead/Write,ReadOnlyandNoAccess.
action
collectionlevel
databaselevel
readadocument
Read/WriteorReadOnly
AdministrateorAccess
createadocument
Read/Write
AdministrateorAccess
modifyadocument
Read/Write
AdministrateorAccess
dropadocument
Read/Write
AdministrateorAccess
truncateacollection
Read/Write
AdministrateorAccess
Example
Forexample,given
adatabaseexample
acollectiondatainthedatabaseexample
auserJohnSmith
IftheuserJohnSmithisassignedtheaccesslevelAccessforthedatabaseexampleandthelevelRead/Writeforthecollectiondata,thenthe
userisallowedtoread,create,modifyordeletedocumentsinthecollectiondata.Buttheuseris,forexample,notallowedtocreate
indexesforthecollectiondatanorcreatenewcollectionsinthedatabaseexample.
GrantingAccessLevels
Accesslevelscanbemanagedviathe[webinterface]orin[arangosh].
Inordertograntanaccessleveltoauser,youcanassignoneofthreeaccesslevelsforeachdatabaseandoneofthreelevelsforeach
collectioninadatabase.Theserveraccesslevelfortheuserfollowsfromthedatabaseaccesslevelinthe _systemdatabase,itis
AdministrateifandonlyifthedatabaseaccesslevelisAdministrate.NotethatthismeansthatdatabaseaccesslevelAccessdoesnotgrant
auserserveraccesslevelAdministrate.
WildcardDatabaseAccessLevel
361
M anagingUsers
Withtheabovedefinition,onemustdefinethedatabaseaccesslevelforalldatabase/userpairsintheserver,whichwouldbeverytedious.
Inordertosimplifythisprocess,itispossibletodefine,forauser,awildcarddatabaseaccesslevel.Thiswildcardisusedifthedatabase
accesslevelisnotexplicitlydefinedforacertaindatabase.
Changingthewildcarddatabaseaccesslevelforauserwillchangetheaccesslevelforalldatabasesthathavenoexplicitlydefinedaccess
level.Notethatthisincludesdatabaseswhichwillbecreatedinthefutureandforwhichnoexplicitaccesslevelsaresetforthatuser!
Ifyoudeletethewildard,thedefaultaccesslevelisdefinedasNoAccess.
Example
AssumeuserJohnSmithhasthefollowingdatabaseaccesslevels:
accesslevel
database *
Access
database shop1
Administrate
database shop2
NoAccess
ThiswillgivetheuserJohnSmiththefollowingdatabaselevelaccess:
database shop1:Administrate
database shop2:NoAccess
database something:Access
Ifthewildcard *ischangedfromAccesstoNoAccessthenthepermissionswillchangeasfollows:
database shop1:Administrate
database shop2:NoAccess
database something:NoAccess
WildcardCollectionAccessLevel
Foreachuseranddatabasethereisawildcardcollectionaccesslevel.Thislevelisusedforallcollectionspairswithoutanexplicitly
definedcollectionaccesslevel.Notethatthisincludescollectionswhichwillbecreatedinthefutureandforwhichnoexplicitaccess
levelsaresetforathatuser!
Ifyoudeletethewildcard,thesystemdefaultstoNoAccess.
Example
AssumeuserJohnSmithhasthefollowingdatabaseaccesslevels:
accesslevel
database *
Access
andcollectionaccesslevels:
accesslevel
database *,collection *
Read/Write
database shop1,collection products
Read-Only
database shop1,collection *
NoAccess
database shop2,collection *
Read-Only
Thentheuserdoewillgetthefollowingcollectionaccesslevels:
database shop1,collection products:Read-Only
database shop1,collection customers:NoAccess
database shop2,collection reviews:Read-Only
database something,collection else:Read/Write
362
M anagingUsers
Explanation:
Database shop1,collection productsdirectlymatchesadefinedaccesslevel.ThislevelisdefinedasRead-Only.
Database shop1,collection customersdoesnotmatchadefinedaccesslevel.However,database shop1matchesandthewildcardin
thisdatabaseforcollectionlevelisNoAccess.
Database shop2,collection reviewsdoesnotmatchadefinedaccesslevel.However,database shop2matchesandthewildcardinthis
databaseforcollectionlevelisRead-Only.
Database somehing,collection elsedoesnotmatchadefinedaccesslevel.Thedatabase somethingalsodoeshaveadirectmatches.
Thereforethewildcardisselected.ThelevelisRead/Write.
SystemCollections
Theaccesslevelforsystemcollectionscannotbechanged.Theyfollowdifferentrulesthanuserdefinedcollectionsandmaychange
withoutfurthernotice.Currentlythesystemcollectionsfollowtheserules:
collection
accesslevel
_users(in_system)
NoAccess
_queues
Read-Only
_frontend
Read/Write
*
sameasdb
AllothersystemcollectionshaveaccesslevelRead/WriteiftheuserhasAdministrateaccesstothedatabase.Theyhaveaccesslevel
Read/OnlyiftheuserhasAccesstothedatabase.
TomodifythesesystemcollectionsyoushouldalwaysusethespecializedAPIsprovidedbyArangoDB.Forexamplenouserhasaccess
tothe_userscollectioninthe_systemdatabase.Allchangestotheaccesslevelsmustbedoneusingthe@arangodb/usersmodule,the
/_users/APIorthewebinterface.
363
InArangosh
ManagingUsersintheArangoDBShell
Pleasenote,thatforbackwardcompatibilitytheserveraccesslevelsfollowfromthedatabaseaccesslevelonthedatabase_system.
Alsonotethattheserveranddatabaseaccesslevelsarerepresentedas
rw:forAdministrate
ro:forAccess
none:forNoaccess
Thisisagainforbackwardcompatibility.
Example
Fireuparangoshandrequiretheusersmodule.Useittocreateanewuser:
arangosh>varusers=require('@arangodb/users');
arangosh>users.save('JohnSmith','mypassword');
CreatesausercalledJohnSmith.Thisuserwillhavenoaccessatall.
arangosh>users.grantDatabase('JohnSmith','testdb','rw');
ThisgrantstheuserAdministrateaccesstothedatabasetestdb. revokeDatabasewillrevokethisaccesslevelsetting.
Note:Beawarethatfrom3.2onwardsthe grantDatabasewillnotautomaticallygrantuserstheaccessleveltowriteorreadcollections
inadatabase.Ifyougrantaccesstoadatabase testdbyouwilladditionallyneedtoexplicitlygrantaccesslevelstoindividual
collectionsvia grantCollection.
Theupgradeprocedurefrom3.1to3.2setsthewildcarddatabaseaccesslevelforalluserstoAdministrateandsetsthewildcard
collectionaccesslevelforalluser/databasepairstoRead/Write.
arangosh>users.grantCollection('JohnSmith','testdb','testcoll','rw');
Save
users.save(user,passwd,active,extra)
ThiswillcreateanewArangoDBuser.Theusernamemustbespecifiedinuserandmustnotbeempty.
Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.Ifyoupassthespecialvalue
ARANGODB_DEFAULT_ROOT_PASSWORD,thepasswordwillbesetthevaluestoredintheenvironmentvariable
ARANGODB_DEFAULT_ROOT_PASSWORD.ThiscanbeusedtopassaninstancevariableintoArangoDB.Forexample,theinstanceidentifierfrom
Amazon.
Iftheactiveattributeisnotspecified,itdefaultstotrue.Theextraattributecanbeusedtosavecustomdatawiththeuser.
Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,ortherealreadyexistsauser
withthespecifiedname.
Note:Theuserwillnothavepermissiontoaccessanydatabase.Youneedtogranttheaccessrightsforoneormoredatabasesusing
grantDatabase.
Examples
arangosh>require('@arangodb/users').save('my-user','my-secret-password');
showexecutionresults
364
InArangosh
GrantDatabase
users.grantDatabase(user,database,type)
Thisgrantstype('rw','ro'or'none')accesstothedatabasefortheuser.Ifdatabaseis "*",thissetsthewildcarddatabaseaccesslevel
fortheuseruser.
Theserveraccesslevelfollowsfromtheaccesslevelforthedatabase _system.
RevokeDatabase
users.revokeDatabase(user,database)
Thisclearstheaccesslevelsettingtothedatabasefortheuserandthewildcarddatabaseaccesssettingforthisuserkicksin.Incaseno
wildcardaccesswasdefinedthedefaultisNoAccess.Thiswillalsocleartheaccesslevelsforallthecollectionsinthisdatabase.
GrantCollection
users.grantCollection(user,database,collection,type)
Thisgrantstype('rw','ro'or'none')accessleveltothecollectionindatabasefortheuser.Ifcollectionis "*"thissetsthewildcard
collectionaccesslevelfortheuseruserindatabasedatabase.
RevokeCollection
users.revokeCollection(user,database)
Thisclearstheaccesslevelsettingtothecollectioncollectionfortheuseruser.Thesystemwilleitherfallbacktothewildcardcollection
accesslevelordefaulttoNoAccess
Replace
users.replace(user,passwd,active,extra)
ThiswilllookupanexistingArangoDBuserandreplaceitsuserdata.
Theusernamemustbespecifiedinuser,andauserwiththespecifiednamemustalreadyexistinthedatabase.
Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.
Iftheactiveattributeisnotspecified,itdefaultstotrue.Theextraattributecanbeusedtosavecustomdatawiththeuser.
Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,orifthespecifiedusercannot
befoundinthedatabase.
Note:thisfunctionwillnotworkfromwithinthewebinterface
Examples
arangosh>require("@arangodb/users").replace("my-user","my-changed-password");
showexecutionresults
Update
users.update(user,passwd,active,extra)
ThiswillupdateanexistingArangoDBuserwithanewpasswordandotherdata.
Theusernamemustbespecifiedinuserandtheusermustalreadyexistinthedatabase.
365
InArangosh
Thepasswordmustbegivenasastring,too,butcanbeleftemptyifrequired.
Iftheactiveattributeisnotspecified,thecurrentvaluesavedfortheuserwillnotbechanged.Thesameistruefortheextraattribute.
Thismethodwillfailifeithertheusernameorthepasswordsarenotspecifiedorgiveninawrongformat,orifthespecifiedusercannot
befoundinthedatabase.
Examples
arangosh>require("@arangodb/users").update("my-user","my-secret-password");
showexecutionresults
isValid
users.isValid(user,password)
Checkswhetherthegivencombinationofusernameandpasswordisvalid.Thefunctionwillreturnabooleanvalueifthecombinationof
usernameandpasswordisvalid.
Eachcalltothisfunctionispenalizedbytheserversleepingarandomamountoftime.
Examples
arangosh>require("@arangodb/users").isValid("my-user","my-secret-password");
true
Remove
users.remove(user)
RemovesanexistingArangoDBuserfromthedatabase.
TheusernamemustbespecifiedinUserandthespecifiedusermustexistinthedatabase.
Thismethodwillfailiftheusercannotbefoundinthedatabase.
Examples
arangosh>require("@arangodb/users").remove("my-user");
Document
users.document(user)
FetchesanexistingArangoDBuserfromthedatabase.
Theusernamemustbespecifiedinuser.
Thismethodwillfailiftheusercannotbefoundinthedatabase.
Examples
arangosh>require("@arangodb/users").document("my-user");
showexecutionresults
All
366
InArangosh
users.all()
FetchesallexistingArangoDBusersfromthedatabase.
Examples
arangosh>require("@arangodb/users").all();
showexecutionresults
Reload
users.reload()
Reloadstheuserauthenticationdataontheserver
Alluserauthenticationdataisloadedbytheserveronceonstartuponlyandiscachedafterthat.Whenusersgetaddedordeleted,acache
flushisdoneautomatically,andthiscanbeperformedbyacalltothismethod.
Examples
arangosh>require("@arangodb/users").reload();
Permission
users.permission(user,database[,collection])
Fetchestheaccessleveltothedatabaseoracollection.
Theuseranddatabasenamemustbespecified,optionallyyoucanspecifythecollectionname.
Thismethodwillfailiftheusercannotbefoundinthedatabase.
Examples
arangosh>require("@arangodb/users").permission("my-user","testdb");
rw
367
ServerConfiguration
Command-lineoptions
GeneralOptions
Generalhelp
--help
-h
Printsalistofthemostcommonoptionsavailableandthenexits.Inordertoseealloptionsuse--help-all.
Version
--version
-v
Printstheversionoftheserverandexits.
ConfigurationFiles
Optionscanbespecifiedonthecommandlineorinconfigurationfiles.IfastringVariableoccursinthevalue,itisreplacedbythe
correspondingenvironmentvariable.
--configurationfilename
-cfilename
Specifiesthenameoftheconfigurationfiletouse.
Ifthiscommandisnotpassedtotheserver,thenbydefault,theserverwillattempttofirstlocateafilenamed~/.arango/arangod.confin
theuser'shomedirectory.
Ifnosuchfileisfound,theserverwillproceedtolookforafilearangod.confinthesystemconfigurationdirectory.Thesystem
configurationdirectoryisplatform-specific,andmaybechangedwhencompilingArangoDByourself.Itmaydefaultto/etc/arangodbor
/usr/local/etc/arangodb.Thisfileisinstalledwhenusingapackagemanagerlikerpmordpkg.Ifyoumodifythisfileandlaterupgradeto
anewversionofArangoDB,thenthepackagemanagernormallywarnsyouabouttheconflict.Inordertoavoidthesewarningforsmall
adjustments,youcanputlocaloverridesintoafilearangod.conf.local.
Onlycommandlineoptionswithavalueshouldbesetwithintheconfigurationfile.Commandlineoptionswhichactasflagsshouldbe
enteredonthecommandlinewhenstartingtheserver.
Whitespaceintheconfigurationfileisignored.Eachoptionisspecifiedonaseparatelineintheform
key=value
Alternatively,aheadersectioncanbespecifiedandoptionspertainingtothatsectioncanbespecifiedinashorterform
[log]
level=trace
ratherthanspecifying
log.level=trace
Soyouseeingeneral --section.paramvaluetranslatesto
[section]
param=value
368
ServerConfiguration
Whereonesectionmayoccurmultipletimes,andthelastoccuranceof paramwillbecomethefinalvalue.Incaseofparametersbeing
vectors,multipleoccuranceaddsanotheritemtothevector.Vectorscanbeidentifiedbythe ...inthe --helpoutputofthebinaries.
Commentscanbeplacedintheconfigurationfile,onlyifthelinebeginswithoneormorehashsymbols(#).
Theremaybeoccasionswhereaconfigurationfileexistsandtheuserwishestooverrideconfigurationsettingsstoredinaconfiguration
file.Anysettingsspecifiedonthecommandlinewilloverwritethesamesettingwhenitappearsinaconfigurationfile.Iftheuserwishes
tocompletelyignoreconfigurationfileswithoutnecessarilydeletingthefile(orfiles),thenaddthecommandlineoption
-cnone
or
--configurationnone
Whenstartinguptheserver.Notethat,thewordnoneiscase-insensitive.
369
OperatingSystemConfiguration
OperatingSystemConfiguration
FileSystems
(LINUX)
WerecommendnottouseBTRFSonlinux,it'sknowntonotworkwellinconjunctionwithArangoDB.Weexperiencedthatarangodb
facinglatencyissuesonaccessingitsdatabasefilesonBTRFSpartitions.InconjunctionwithBTRFSandAUFSwealsosawdataloss
onrestart.
VirtualMemoryPageSizes
(LINUX)
Bydefault,ArangoDBusesJemallocasthememoryallocator.Jemallocdoesagoodjobofreducingvirtualmemoryfragmentation,
especiallyforlong-runningprocesses.Unfortunately,someOSconfigurationscaninterferewithJemalloc'sabilitytofunctionproperly.
Specifically,Linux's"transparenthugepages",Windows'"largepages"andothersimilarfeaturessometimespreventJemallocfrom
returningunusedmemorytotheoperatingsystemandresultinunnecessarilyhighmemoryuse.Therefore,werecommenddisablingthese
featureswhenusingJemallocwithArangoDB.Pleaseconsultyouroperatingsystem'sdocumentationforhowtodothis.
Execute
sudobash-c"echomadvise>/sys/kernel/mm/transparent_hugepage/enabled"
sudobash-c"echomadvise>/sys/kernel/mm/transparent_hugepage/defrag"
beforeexecuting arangod.
SwapSpace
(LINUX)
Itisrecommendedtoassignswapspaceforaserverthatisrunningarangod.Configuringswapspacecanpreventtheoperatingsystem's
OOM killerfromkillingArangoDBtooeagerlyonLinux.
Over-CommitMemory
Execute
sudobash-c"echo0>/proc/sys/vm/overcommit_memory"
beforeexecuting arangod.
Fromwww.kernel.org:
Whenthisflagis0,thekernelattemptstoestimatetheamountoffreememoryleftwhenuserspacerequestsmorememory.
Whenthisflagis1,thekernelpretendsthereisalwaysenoughmemoryuntilitactuallyrunsout.
Whenthisflagis2,thekernelusesa"neverovercommit"policythatattemptstopreventanyovercommitofmemory.
ZoneReclaim
Execute
sudobash-c"echo0>/proc/sys/vm/overcommit_memory"
370
OperatingSystemConfiguration
beforeexecuting arangod.
Fromwww.kernel.org:
ThisisvalueORedtogetherof
1=Zonereclaimon
2=Zonereclaimwritesdirtypagesout
4=Zonereclaimswapspages
NUMA
M ulti-processorsystemsoftenhavenon-uniformAccessM emory(NUM A).ArangoDBshouldbestartedwithinterleaveonsuch
system.Thiscanbeachievedusing
numactl--interleave=allarangod...
EnvironmentVariables
Itisrecommendedtosettheenvironmentvariable GLIBCXX_FORCE_NEWto1onsystemsthatuseglibc++inordertodisablethememory
poolingbuiltintoglibc++.ThatmemorypoolingisunnecessarybecauseJemallocwillalreadydomemorypooling.
Execute
exportGLIBCXX_FORCE_NEW=1
beforestarting arangod.
32bit
WhileitispossibletocompileArangoDBon32bitsystem,thisisnotarecommendedenvironment.64bitsystemscanaddressa
significantlybiggermemoryregion.
371
M anagingEndpoints
ManagingEndpoints
TheArangoDBservercanlistenforincomingrequestsonmultipleendpoints.
TheendpointsarenormallyspecifiedeitherinArangoDB'sconfigurationfileoronthecommand-line,usingthe --server.endpoint.
ArangoDBsupportsdifferenttypesofendpoints:
tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4
tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6
ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryption
ssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryption
unix:///path/to/socket-Unixdomainsocketendpoint
IfaTCP/IPendpointisspecifiedwithoutaportnumber,thenthedefaultport(8529)willbeused.Ifmultipleendpointsneedtobeused,
theoptioncanberepeatedmultipletimes.
ThedefaultendpointforArangoDBistcp://127.0.0.1:8529ortcp://localhost:8529.
EXAMPLES
unix>./arangod--server.endpointtcp://127.0.0.1:8529
--server.endpointssl://127.0.0.1:8530
--ssl.keyfileserver.pem/tmp/vocbase
2012-07-26T07:07:47Z[8161]INFOusingSSLprotocolversion'TLSv1'
2012-07-26T07:07:48Z[8161]INFOusingendpoint'ssl://127.0.0.1:8530'forhttpsslrequests
2012-07-26T07:07:48Z[8161]INFOusingendpoint'tcp://127.0.0.1:8529'forhttptcprequests
2012-07-26T07:07:49Z[8161]INFOArangoDB(version1.1.alpha)isreadyforbusiness
2012-07-26T07:07:49Z[8161]INFOHaveFun!
TCPEndpoints
Givenahostname:
--server.endpointtcp://hostname:port
GivenanIPv4address:
--server.endpointtcp://ipv4-address:port
GivenanIPv6address:
--server.endpointtcp://[ipv6-address]:port
Ononespecificethernetinterfaceeachportcanonlybeboundexactlyonce.Youcanlookupyouravailableinterfacesusingtheifconfig
commandonLinux/M acOSX-theWindowsequivalentisipconfig(SeeWikipediaformoredetails).Thegeneralnamesoftheinterfaces
differonOS'sandhardwarestheyrunon.However,typicallyeveryhosthasasocalledloopbackinterface,whichisavirtualinterface.
Byconventionitalwayshastheaddress127.0.0.1or::1(ipv6),andcanonlybereachedfromexactlytheverysamehost.Ethernet
interfacesusuallyhavenameslikeeth0,wlan0,eth1:17,le0oraplaintextnameinWindows.
Tofindoutwhichservicesalreadyuseports(soArangoDBcan'tbindthemanymore),youcanusethenetstatcommand(itbehavesa
littledifferentoneachplatform,runitwith-lnptonLinux,-ptcponM acOSXorwith-anonwindowsforvaluableinformation).
ArangoDBcanalsodoasocalledbroadcastbindusingtcp://0.0.0.0:8529.Thiswayitwillbereachableonallinterfacesofthehost.This
maybeusefulondevelopmentsystemsthatfrequentlychangetheirnetworksetuplikelaptops.
SpecialnoteonIPv6link-localaddresses
ArangoDBcanalsolistentoIPv6link-localaddressesviaaddingthezoneIDtotheIPv6addressintheform [ipv6-link-localaddress%zone-id].However,whatyouprobablyinsteadwantistobindtoalocalIPv6address.LocalIPv6addressesstartwith fd.If
youonlyseea fe80:IPv6addressinyourinterfaceconfigurationbutnoIPv6addressstartingwith fdyourinterfacehasnolocal
IPv6addressassigned.YoucanreadmoreaboutIPv6link-localaddresseshere.
Example
372
M anagingEndpoints
Bindtoalink-localandlocalIPv6address.
unix>ifconfig
Thiscommandlistsallinterfacesandassignedipaddresses.Thelink-localaddressmaybe fe80::6257:18ff:fe82:3ec6%eth0(IPv6
addressplusinterfacename).AlocalIPv6addressmaybe fd12:3456::789a.TobindArangoDBtoitstartarangodwith -server.endpointtcp://[fe80::6257:18ff:fe82:3ec6%eth0]:8529.Usetelnettotesttheconnection.
unix>telnetfe80::6257:18ff:fe82:3ec6%eth08529
Tryingfe80::6257:18ff:fe82:3ec6...
Connectedtomy-machine.
Escapecharacteris'^]'.
GET/HTTP/1.1
HTTP/1.1301MovedPermanently
Location:/_db/_system/_admin/aardvark/index.html
Content-Type:text/html
Server:ArangoDB
Connection:Keep-Alive
Content-Length:197
<html><head><title>Moved</title></head><body><h1>Moved</h1><p>Thispagehasmovedto<ahref="/_db/_system/_admin/aardvark/inde
x.html">/_db/_system/_admin/aardvark/index.html</a>.</p></body></html>
Reuseaddress
--tcp.reuse-address
IfthisbooleanoptionissettotruethenthesocketoptionSO_REUSEADDRissetonallserverendpoints,whichisthedefault.Ifthis
optionissettofalseitispossiblethatittakesuptoaminuteafteraserverhasterminateduntilitispossibleforanewservertousethe
sameendpointagain.Thisiswhythisisactivatedbydefault.
Pleasenotehoweverthatundersomeoperatingsystemsthiscanbeasecurityriskbecauseitmightbepossibleforanotherprocessto
bindtothesameaddressandport,possiblyhijackingnetworktraffic.UnderWindows,ArangoDBadditionallysetstheflag
SO_EXCLUSIVEADDRUSEasameasuretoalleviatethisproblem.
Backlogsize
--tcp.backlog-size
AllowstospecifythesizeofthebacklogforthelistensystemcallThedefaultvalueis10.Themaximumvalueisplatform-dependent.
Specifyingahighervaluethandefinedinthesystemheader'sSOM AXCONNmayresultinawarningonserverstart.Theactualvalue
usedbylistenmayalsobesilentlytruncatedonsomeplatforms(thishappensinsidethelistensystemcall).
373
SSLConfiguration
SSLConfiguration
SSLEndpoints
Givenahostname:
--server.endpointtcp://hostname:port
GivenanIPv4address:
--server.endpointtcp://ipv4-address:port
GivenanIPv6address:
--server.endpointtcp://[ipv6-address]:port
Note:IfyouareusingSSL-encryptedendpoints,youmustalsosupplythepathtoaservercertificateusingthe --ssl.keyfileoption.
Keyfile
--ssl.keyfilefilename
IfSSLencryptionisused,thisoptionmustbeusedtospecifythefilenameoftheserverprivatekey.ThefilemustbePEM formatted
andcontainboththecertificateandtheserver'sprivatekey.
Thefilespecifiedbyfilenamecanbegeneratedusingopenssl:
#createprivatekeyinfile"server.key"
opensslgenrsa-des3-outserver.key1024
#createcertificatesigningrequest(csr)infile"server.csr"
opensslreq-new-keyserver.key-outserver.csr
#copyawayoriginalprivatekeyto"server.key.org"
cpserver.keyserver.key.org
#removepassphrasefromtheprivatekey
opensslrsa-inserver.key.org-outserver.key
#signthecsrwiththekey,createscertificatePEMfile"server.crt"
opensslx509-req-days365-inserver.csr-signkeyserver.key-outserver.crt
#combinecertificateandkeyintosinglePEMfile"server.pem"
catserver.crtserver.key>server.pem
YoumayusecertificatesissuedbyaCertificateAuthorityorself-signedcertificates.Self-signedcertificatescanbecreatedbyatoolof
yourchoice.WhenusingOpenSSLforcreatingtheself-signedcertificate,thefollowingcommandsshouldcreateavalidkeyfile:
-----BEGINCERTIFICATE----(base64encodedcertificate)
-----ENDCERTIFICATE---------BEGINRSAPRIVATEKEY----(base64encodedprivatekey)
-----ENDRSAPRIVATEKEY-----
Forfurtherinformationpleasecheckthemanualsofthetoolsyouusetocreatethecertificate.
CAFile
--ssl.cafilefilename
374
SSLConfiguration
ThisoptioncanbeusedtospecifyafilewithCAcertificatesthataresenttotheclientwhenevertheserverrequestsaclientcertificate.If
thefileisspecified,TheserverwillonlyacceptclientrequestswithcertificatesissuedbytheseCAs.Donotspecifythisoptionifyou
wantclientstobeabletoconnectwithoutspecificcertificates.
ThecertificatesinfilenamemustbePEM formatted.
SSLprotocol
--ssl.protocolvalue
Usethisoptiontospecifythedefaultencryptionprotocoltobeused.Thefollowingvariantsareavailable:
1:SSLv2
2:SSLv2orSSLv3(negotiated)
3:SSLv3
4:TLSv1
5:TLSv1.2
Thedefaultvalueis5(TLSv1.2).
SSLcache
--ssl.session-cachevalue
SettotrueifSSLsessioncachingshouldbeused.
valuehasadefaultvalueoffalse(i.e.nocaching).
SSLpeercertificate
ThisfeatureisavailableintheEnterpriseEdition.
--ssl.require-peer-certificate
Requireapeercertificatefromtheclientbeforeconnecting.
SSLoptions
--ssl.optionsvalue
ThisoptioncanbeusedtosetvariousSSL-relatedoptions.IndividualoptionvaluesmustbecombinedusingbitwiseOR.
WhichoptionsareavailableonyourplatformisdeterminedbytheOpenSSLversionyouuse.Thelistofoptionsavailableonyour
platformmightberetrievedbythefollowingshellcommand:
>grep"#defineSSL_OP_.*"/usr/include/openssl/ssl.h
#defineSSL_OP_MICROSOFT_SESS_ID_BUG0x00000001L
#defineSSL_OP_NETSCAPE_CHALLENGE_BUG0x00000002L
#defineSSL_OP_LEGACY_SERVER_CONNECT0x00000004L
#defineSSL_OP_NETSCAPE_REUSE_CIPHER_CHANGE_BUG0x00000008L
#defineSSL_OP_SSLREF2_REUSE_CERT_TYPE_BUG0x00000010L
#defineSSL_OP_MICROSOFT_BIG_SSLV3_BUFFER0x00000020L
...
AdescriptionoftheoptionscanbefoundonlineintheOpenSSLdocumentation
SSLcipher
--ssl.cipher-listcipher-list
ThisoptioncanbeusedtorestricttheservertocertainSSLciphersonly,andtodefinetherelativeusagepreferenceofSSLciphers.
Theformatofcipher-listisdocumentedintheOpenSSLdocumentation.
Tocheckwhichciphersareavailableonyourplatform,youmayusethefollowingshellcommand:
375
SSLConfiguration
>opensslciphers-v
ECDHE-RSA-AES256-SHASSLv3Kx=ECDHAu=RSAEnc=AES(256)Mac=SHA1
ECDHE-ECDSA-AES256-SHASSLv3Kx=ECDHAu=ECDSAEnc=AES(256)Mac=SHA1
DHE-RSA-AES256-SHASSLv3Kx=DHAu=RSAEnc=AES(256)Mac=SHA1
DHE-DSS-AES256-SHASSLv3Kx=DHAu=DSSEnc=AES(256)Mac=SHA1
DHE-RSA-CAMELLIA256-SHASSLv3Kx=DHAu=RSAEnc=Camellia(256)
Mac=SHA1
...
Thedefaultvalueforcipher-listis"ALL".
376
LDAPOptions
LDAP
ThisfeatureisavailableintheEnterpriseEdition.
Thebasicoptionsare --ldap.enabled, --ldap.tls, --ldap.port, --ldap.serverand --ldap.permissions-attribute-name.
--ldap.serverand --ldap.portcanbereplaceby --ldap.url.
Thedefaultfor --ldap.portis389.
--ldap.permissions-attribute-namehastheformatdatabse-name=(*|rw|none)[,database-name=(*|rw|none)].
Therearetwomodesofoperation:simpleauthandbind+search.
simpleauth
ArangoDBconnectstotheldapserverandauthenticateswiththeusernameandpasswordprovidedbytheapiauthenticationrequestand
searchesforthedatabasepermissionsusingtheattributenameprovidedby --ldap.permissions-attribute-name.
Example:
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.permissions-attribute-namearangodbPermissions\
--ldap.prefixuid=--ldap.suffix,dc=company,dc=com
--ldap.prefixand --ldap.suffixbuildthedistinguishedname(DN).ArangoDBtrystoauthenticatewithprefix+ArangoDB
username+suffixagainsttheldapserverandsearchesforthedatabasepermissions.
bind+search
Examplewithanonymousauth:
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.basedndc=company,dc=com\
--ldap.permissions-attribute-namearangodbPermissions
WiththisconfigurationArangoDBbindsanonymouslytotheldapserverandsearchesfortheuser.Iftheuserisfoundaauthenticationis
donewiththeusersDNandpasswordandthendatabasepermissionsarefetched.
ExamplewithDNandpassword:
--ldap.enabledtrue--ldap.serverldap.company.com\
--ldap.basedndc=company,dc=com\
--ldap.permissions-attribute-namearangodbPermissions
--ldap.binddncn=admin,dc=company,dc=com--ldap.bindpasswdadmin
WiththisconfigurationArangoDBbindswith --ldap.bindnand --ldap.bindpasswdtotheldapserverandsearchesfortheuser.Ifthe
userisfoundaauthenticationisdonewiththeusersDNandpasswordandthendatabasepermissionsarefetched.
additionaloptions
--ldap.search-filter"objectClass=*"
Restrictthesearchtospecificobjectclasses.Thedefaultis objectClass=*.
--ldap.search-attribute"uid"
--ldap.search-attributespecifieswhichattributetocomparewiththeusername.Thedefaultis uid.
377
LDAPOptions
--ldap.search-scopesub
`--ldap.search-scopespecifiesinwhichscopetosearchforauser.Validareoneofbase,oneorsub.Thedefaultissub.
ldapurl
--ldap.urlldap://ldap.server.com:1234/dc=example,dc=com?uid?sub
Theldapurlconsistsoftheldapserverandport,abasedn,asearchattributeandascopewhichcanbeoneofbase,oneorsub.
TLSoptions
Aencryptedconnectioncanbeestablishedwith --ldap.tlstrueunderUNIXandGNU/Linuxplatforms.
AllfollowingoptionsarenotavailableunderWindows.
--ldap.tls
Thedefaultis false.With trueatlsconnectionisestablished.
--ldap.tls-version
Thedefaultis 1.2.Availableversionsare 1.0, 1.1and 1.2.
--ldap.tls-cert-check-strategy
Thedefaultis hard.Availablestrategiesare never, hard, demand, allowand try.
--ldap.tls-cacert-file
Afilepathtooneormore(concatenated)certificateauthoritycertificatesinpemformat.Asdefaultnofilepathisconfigured.
Followingoptionhasnoeffect/doesnotworkundermacOS.
--ldap.tls-cacert-dir
Adirectorypathtocertificateauthoritycertificatesinc_rehashformat.Asdefaultnodirectorypathisconfigured.
378
LoggingOptions
Command-LineOptionsforLogging
Loglevelsandtopics
ArangoDB'slogoutputisgroupedintotopics. --log.levelcanbespecifiedmultipletimesatstartup,forasmanytopicsasneeded.
Thelogverbosityandoutputfilescanbeadjustedperlogtopic.Forexample
--log.levelstartup=trace--log.levelqueries=trace--log.levelinfo
willlogmessagesconcerningstartupattracelevel,AQLqueriesattracelevelandeverythingelseatinfolevel.
Inaconfigurationfile,itiswrittenlikethis:
[log]
level=startup=trace
level=queries=trace
level=info
Notethattheremustnotbeanywhitespacearoundthesecond =.
Theavailableloglevelsare:
fatal:onlylogsfatalerrors
error:onlylogserrors
warning:onlylogswarningsanderrors
info:logsinformationmessages,warningsanderrors
debug:logsdebugandinformationmessages,warningsanderrors
trace:logstrace,debugandinformationmessages,warningsanderrors
Notethatlevels debugand tracewillbeveryverbose.
SomerelevantlogtopicsavailableinArangoDB3are:
agency:informationabouttheagency
collector:informationabouttheWALcollector'sstate
compactor:informationaboutthecollectiondatafilecompactor
datafiles:datafile-relatedoperations
mmap:informationaboutmemory-mappingoperations(includingmsync)
performance:performance-releatedmessages
queries:executedAQLqueries,slowqueries
replication:replication-relatedinfo
requests:HTTPrequests
startup:informationaboutserverstartupandshutdown
threads:informationaboutthreads
Logoutputs
Thelogoption --log.output<definition>allowsdirectingtheglobalorper-topiclogoutputtodifferentoutputs.Theoutputdefinition
<definition>canbeoneof
-forstdin
+forstderr
syslog://<syslog-facility>
syslog://<syslog-facility>/<application-name>
file://<relative-path>
Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputfordifferentlogtopics.Tosetupaper-topicoutput
configuration,use --log.output<topic>=<definition>,e.g.
379
LoggingOptions
queries=file://queries.txt
logsallqueriestothefile"queries.txt".
Theoldoption --log.fileisstillavailablein3.0forconveniencereasons.In3.0itisashortcutforthemoregeneraloption -log.outputfile://filename.
Theoldoption --log.requests-fileisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption --log.output
requests=file://....
Using --log.outputalsoallowsdirectinglogoutputtodifferentfilesbasedontopics.Forexample,tologallAQLqueriestoafile
"queries.log"onecanusetheoptions:
--log.levelqueries=trace--log.outputqueries=file:///path/to/queries.log
ToadditionallylogHTTPrequesttoafilenamed"requests.log"addtheoptions:
--log.levelrequests=info--log.outputrequests=file:///path/to/requests.log
Forcingdirectoutput
Theoption --log.force-directcanbeusedtodisablelogginginanextraloggingthread.Ifsetto true,anylogmessagesare
immediatelyprintedinthethreadthattriggeredthelogmessage.Thisisnon-optimalforperformancebutcanaiddebugging.Ifsetto
false,logmessagesarehandedofftoanextraloggingthread,whichasynchronouslywritesthelogmessages.
Localtime
Logdatesandtimesinlocaltimezone: --log.use-local-time
Ifspecified,alldatesandtimesinlogmessageswillusetheserver'slocaltime-zone.Ifnotspecified,alldatesandtimesinlogmessages
willbeprintedinUTC/Zulutime.Thedateandtimeformatusedinlogsisalways YYYY-MM-DDHH:MM:SS,regardlessofthissetting.If
UTCtimeisused,a ZwillbeappendedtoindicateZulutime.
Linenumber
Loglinenumber: --log.line-number
Normally,ifanhumanreadablefatal,error,warningorinfomessageislogged,noinformationaboutthefileandlinenumberisprovided.
Thefileandlinenumberisonlyloggedfordebugandtracemessage.Thisoptioncanbeusetoalwayslogthesepiecesofinformation.
Prefix
Logprefix: --log.prefixprefix
Thisoptionisusedspecifyanprefixtologgedtext.
Thread
Logthreadidentifier: --log.thread
Wheneverlogoutputisgenerated,theprocessIDiswrittenaspartoftheloginformation.Settingthisoptionappendsthethreadidof
thecallingthreadtotheprocessid.Forexample,
2010-09-20T13:04:01Z[19355]INFOreadyforbusiness
whennothreadisloggedand
2010-09-20T13:04:17Z[19371-18446744072487317056]readyforbusiness
whenthiscommandlineoptionisset.
380
LoggingOptions
381
GeneralOptions
GeneralOptions
DatabaseUpgrade
--database.auto-upgrade
Specifyingthisoptionwillmaketheserverperformadatabaseupgradeatstart.Adatabaseupgradewillfirstcomparetheversionnumber
storedinthefileVERSIONinthedatabasedirectorywiththecurrentserverversion.
Ifthetwoversionnumbersmatch,theserverwillstartnormally.
Iftheversionnumberfoundinthedatabasedirectoryishigherthantheversionnumbertheserverisrunning,theserverexpectsthisisan
unintentionaldowngradeandwillwarnaboutthis.Itwillhoweverstartnormally.Usingtheserverintheseconditionsishowevernot
recommendednorsupported.
Iftheversionnumberfoundinthedatabasedirectoryislowerthantheversionnumbertheserverisrunning,theserverwillcheck
whetherthereareanyupgradetaskstoperform.Itwillthenexecuteallrequiredupgradetasksandprinttheirstatuses.Ifoneofthe
upgradetasksfails,theserverwillexitandrefusetostart.Re-startingtheserverwiththeupgradeoptionwillthenagaintriggerthe
upgradecheckandexecutionuntiltheproblemisfixed.Ifalltasksarefinished,theserverwillstartnormally.
Whetherornotthisoptionisspecified,theserverwillalwaysperformaversioncheckonstartup.RunningtheserverwithanonmatchingversionnumberintheVERSIONfilewillmaketheserverrefusetostart.
StorageEngine
AsofArangoDB3.2twostorageenginesaresupported.The"traditional"engineiscalled MMFiles,whichisalsothedefaultstorage
engine.
AnalternativeenginebasedonRocksDBisalsoprovidedandcanbeturnedonmanually.
Onestorageenginetypeissupportedperserverperinstallation.Liveswitchingofstorageenginesonalreadyinstalledsystemsisn't
supported.Configuringthewrongengine(notmatchingthepreviouslyusedone)willresultintheserverrefusingtostart.Youmay
howeveruse autotoletArangoDBchoosethepreviouslyusedone.
--server.storage-engine[auto|mmfiles|rocksdb]
Daemon
--daemon
Runstheserverasadaemon(asabackgroundprocess).Thisparametercanonlybesetifthepid(processid)fileisspecified.Thatis,
unlessavaluetotheparameterpid-fileisgiven,thentheserverwillreportanerrorandexit.
DefaultLanguage
--default-languagedefault-language
Thedefaultlanguageistusedforsortingandcomparingstrings.Thelanguagevalueisatwo-letterlanguagecode(ISO-639)oritis
composedbyatwo-letterlanguagecodewithandatwolettercountrycode(ISO-3166).Validlanguagesare"de","en","en_US"or
"en_UK".
Thedefaultdefault-languageissettobethesystemlocaleonthatplatform.
Supervisor
--supervisor
Executestheserverinsupervisormode.Intheeventthattheserverunexpectedlyterminatesduetoaninternalerror,thesupervisorwill
automaticallyrestarttheserver.Settingthisflagautomaticallyimpliesthattheserverwillrunasadaemon.Notethat,aswiththedaemon
flag,thisflagrequiresthatthepid-fileparameterwillset.
unix>./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
2012-06-27T15:58:28Z[10133]INFOstartingupinsupervisormode
382
GeneralOptions
Ascanbeseen(e.g.byexecutingthepscommand),thiswillstartasupervisorprocessandtheactualdatabaseprocess:
unix>psfax|greparangod
10137?Ssl0:00./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
10142?Sl0:00\_./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
Whenthedatabaseprocessterminatesunexpectedly,thesupervisorprocesswillstartupanewdatabaseprocess:
>kill-SIGSEGV10142
>psfax|greparangod
10137?Ssl0:00./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
10168?Sl0:00\_./arangod--supervisor--pid-file/var/run/arangodb.pid/tmp/vocbase/
Useridentity
--uiduid
Thename(identity)oftheusertheserverwillrunas.Ifthisparameterisnotspecified,theserverwillnotattempttochangeitsUID,so
thattheUIDusedbytheserverwillbethesameastheUIDoftheuserwhostartedtheserver.Ifthisparameterisspecified,thenthe
serverwillchangeitsUIDafteropeningportsandreadingconfigurationfiles,butbeforeacceptingconnectionsoropeningotherfiles
(suchasrecoveryfiles).Thisisusefulwhentheservermustbestartedwithraisedprivileges(incertainenvironments)butsecurity
considerationsrequirethattheseprivilegesbedroppedoncetheserverhasstartedwork.
Observethatthisparametercannotbeusedtobypassoperatingsystemsecurity.Ingeneral,thisparameter(anditscorresponding
relativegid)canlowerprivilegesbutnotraisethem.
Groupidentity
--gidgid
Thename(identity)ofthegrouptheserverwillrunas.Ifthisparameterisnotspecified,thentheserverwillnotattempttochangeits
GID,sothattheGIDtheserverrunsaswillbetheprimarygroupoftheuserwhostartedtheserver.Ifthisparameterisspecified,then
theserverwillchangeitsGIDafteropeningportsandreadingconfigurationfiles,butbeforeacceptingconnectionsoropeningotherfiles
(suchasrecoveryfiles).
Thisparameterisrelatedtotheparameteruid.
Processidentity
--pid-filefilename
ThenameoftheprocessIDfiletousewhenrunningtheserverasadaemon.Thisparametermustbespecifiedifeithertheflagdaemon
orsupervisorisset.
Console
--console
Runstheserverinanexclusiveemergencyconsolemode.Whenstartingtheserverwiththisoption,theserverisstartedwithan
interactiveJavaScriptemergencyconsole,withallnetworkingandHTTPinterfacesoftheserverdisabled.
Norequestscanbemadetotheserverinthismode,andtheonlywaytoworkwiththeserverinthismodeisbyusingtheemergency
console.Notethattheservercannotbestartedinthismodeifitisalreadyrunninginthisoranothermode.
RandomGenerator
--random.generatorarg
Theargumentisaninteger(1,2,3or4)whichsetsthemannerinwhichrandomnumbersaregenerated.Thedefaultmethod(3)istouse
theanon-blockingrandom(orpseudorandom)numbergeneratorsuppliedbytheoperatingsystem.
383
GeneralOptions
Specifyinganargumentof2,usesablockingrandom(orpseudorandom)numbergenerator.Specifyinganargument1setsa
pseudorandomnumbergeneratorusinganimplicationoftheM ersenneTwisterM T19937algorithm.Algorithm4isacombinationofthe
blockingrandomnumbergeneratorandtheM ersenneTwister.
Enable/disableauthentication
--server.authenticationSettingthisoptiontofalsewillturnoffauthenticationontheserversidesoallclientscanexecuteanyaction
withoutauthorizationandprivilegechecks.Thedefaultvalueistrue.
JWTSecret
--server.jwt-secretsecret
ArangoDBwilluseJWTstoauthenticaterequests.UsingthisoptionletsyouspecifyaJWT.
InsingleserversetupsandwhennotspecifyingthissecretArangoDBwillgenerateasecret.
Inclusterdeploymentswhichhaveauthenticationenabledasecretmustbesetconsistentlyacrossallclustertaskssotheycantalkto
eachother.
Enable/disableauthenticationforUNIXdomainsockets
--server.authentication-unix-socketsvalue
SettingvaluetotruewillturnoffauthenticationontheserversideforrequestscominginviaUNIXdomainsockets.Withthisflag
enabled,clientslocatedonthesamehostastheArangoDBservercanuseUNIXdomainsocketstoconnecttotheserverwithout
authentication.Requestscominginbyothermeans(e.g.TCP/IP)arenotaffectedbythisoption.
Thedefaultvalueisfalse.
Note:thisoptionisonlyavailableonplatformsthatsupportUNIXdomainsockets.
Enable/disableauthenticationforsystemAPIrequestsonly
--server.authentication-system-onlybooleanControlswhetherincomingrequestsneedauthenticationonlyiftheyaredirectedtothe
ArangoDB'sinternalAPIsandfeatures,locatedat/_api/,/_admin/etc.Iftheflagissettotrue,thenHTTPauthenticationisonlyrequired
forrequestsgoingtoURLsstartingwith/_,butnotforotherURLs.Theflagcanthusbeusedtoexposeauser-madeAPIwithoutHTTP
authenticationtotheoutsideworld,buttopreventtheoutsideworldfromusingtheArangoDBAPIandtheadmininterfacewithout
authentication.NotethatcheckingtheURLisperformedafteranydatabasenameprefixhasbeenremoved.Thatmeanswhentheactual
URLcalledis/_db/_system/myapp/myaction,theURL/myapp/myactionwillbeusedforauthentication-system-onlycheck.Thedefaultis
true.NotethatauthenticationstillneedstobeenabledfortheserverregularlyinorderforHTTPauthenticationtobeforcedforthe
ArangoDBAPIandthewebinterface.Settingonlythisflagisnotenough.YoucancontrolArangoDB'sgeneralauthenticationfeature
withthe--server.authenticationflag.
Enable/disablereplicationapplier
--database.replication-applierflag
Iffalsetheserverwillstartwithreplicationappliersturnedoff,evenifthereplicationappliersareconfiguredwiththeautoStartoption.
Usingthecommand-lineoptionwillnotchangethevalueoftheautoStartoptionintheapplierconfiguration,butwillsuppressautostartingthereplicationapplierjustonce.
Iftheoptionisnotused,ArangoDBwillreadtheapplierconfigurationfromthefileREPLICATION-APPLIER-CONFIGonstartup,and
usethevalueoftheautoStartattributefromthisfile.
Thedefaultistrue.
Keep-alivetimeout
--http.keep-alive-timeout
384
GeneralOptions
AllowstospecifythetimeoutforHTTPkeep-aliveconnections.Thetimeoutvaluemustbespecifiedinseconds.Idlekeep-alive
connectionswillbeclosedbytheserverautomaticallywhenthetimeoutisreached.Akeep-alive-timeoutvalue0willdisablethekeep
alivefeatureentirely.
HideProductheader
--http.hide-product-header
Iftrue,theserverwillexcludetheHTTPheader"Server:ArangoDB"inHTTPresponses.Ifsettofalse,theserverwillsendtheheaderin
responses.
Thedefaultisfalse.
Allowmethodoverride
--http.allow-method-override
Whenthisoptionissettotrue,theHTTPrequestmethodwilloptionallybefetchedfromoneofthefollowingHTTPrequestheadersif
presentintherequest:
x-http-method
x-http-method-override
x-method-override
Iftheoptionissettotrueandanyoftheseheadersisset,therequestmethodwillbeoverriddenbythevalueoftheheader.Forexample,
thisallowsissuinganHTTPDELETErequestwhichtotheoutsideworldwilllooklikeanHTTPGETrequest.Thisallowsbypassing
proxiesandtoolsthatwillonlyletcertainrequesttypespass.
Settingthisoptiontotruemayimposeasecurityrisksoitshouldonlybeusedincontrolledenvironments.
Thedefaultvalueforthisoptionisfalse.
Serverthreads
--server.threadsnumber
Specifiesthenumberofthreadsthatarespawnedtohandlerequests.
Togglingserverstatistics
--server.statisticsvalue
Ifthisoptionisvalueisfalse,thenArangoDB'sstatisticsgatheringisturnedoff.StatisticsgatheringcausesregularCPUactivitysousing
thisoptiontoturnitoffmightrelieveheavy-loadedinstancesabit.
Sessiontimeout
timetoliveforserversessions --server.session-timeoutvalue
Thetimeoutforwebinterfacesessions,usingforauthenticatingrequeststothewebinterface(/_admin/aardvark)andrelatedareas.
Sessionsareonlyusedwhenauthenticationisturnedon.
Foxxqueues
enableordisabletheFoxxqueuesfeature --foxx.queuesflagIftrue,theFoxxqueueswillbeavailableandjobsinthequeueswillbe
executedasynchronously.Thedefaultistrue.Whensetto falsethequeuemanagerwillbedisabledandanyjobsarepreventedfrom
beingprocessed,whichmayreduceCPUloadabit.
Foxxqueuespollinterval
pollintervalforFoxxqueues --foxx.queues-poll-intervalvalueThepollintervalfortheFoxxqueuesmanager.Thevalueisspecifiedin
seconds.LowervalueswillmeanmoreimmediateandmorefrequentFoxxqueuejobexecution,butwillmakethequeuethreadwakeup
andquerythequeuesmoreoften.Whensettoalowvalue,thequeuethreadmightcauseCPUload.Thedefaultis1second.IfFoxx
385
GeneralOptions
queuesarenotusedmuch,thenthisvaluemaybeincreasedtomakethequeuesthreadwakeupless.
Directory
--database.directorydirectory
Thedirectorycontainingthecollectionsanddatafiles.Defaultsto/var/lib/arango.Whenspecifyingthedatabasedirectory,pleasemake
surethedirectoryisactuallywritablebythearangodprocess.
YoushouldfurthernotuseadatabasedirectorywhichisprovidedbyanetworkfilesystemsuchasNFS.Thereasonisthatnetworked
filesystemsmightcauseinconsistencieswhentherearemultipleparallelreadersorwritersortheylackfeaturesrequiredbyarangod(e.g.
flock()).
directory
Whenusingthecommandlineversion,youcansimplysupplythedatabasedirectoryasargument.
Examples
>./arangod--server.endpointtcp://127.0.0.1:8529--database.directory
/tmp/vocbase
Journalsize
--database.maximal-journal-sizesizeM aximalsizeofjournalinbytes.Canbeoverwrittenwhencreatinganewcollection.Notethat
thisalsolimitsthemaximalsizeofasingledocument.Thedefaultis32MB.
Waitforsync
defaultwaitforsyncbehavior --database.wait-for-syncbooleanDefaultwait-for-syncvalue.Canbeoverwrittenwhencreatinganew
collection.Thedefaultisfalse.
Forcesyncingofproperties
forcesyncingofcollectionpropertiestodisk --database.force-sync-propertiesbooleanForcesyncingofcollectionpropertiestodisk
aftercreatingacollectionorupdatingitsproperties.Ifturnedoff,nofsyncwillhappenforthecollectionanddatabasepropertiesstored
in parameter.jsonfilesinthefilesystem.Turningoffthisoptionwillspeedupworkloadsthatcreateanddropalotofcollections(e.g.
testsuites).Thedefaultistrue.
LimitingmemoryforAQLqueries
--query.memory-limitvalue
Thedefaultmaximumamountofmemory(inbytes)thatasingleAQLquerycanuse.WhenasingleAQLqueryreachesthespecified
limitvalue,thequerywillbeabortedwitharesourcelimitexceededexception.Inacluster,thememoryaccountingisdonepershard,so
thelimitvalueiseffectivelyamemorylimitperquerypershard.
ThegloballimitvaluecanbeoverridenperquerybysettingthememoryLimitoptionvalueforindividualquerieswhenrunninganAQL
query.
Thedefaultvalueis0,meaningthatthereisnomemorylimit.
TurningAQLwarningsintoerrors
--query.fail-on-warningvalue
Whensettotrue,AQLqueriesthatproducewarningswillinstantlyabortandthrowanexception.Thisoptioncanbesettocatch
obviousissueswithAQLqueriesearly.Whensettofalse,AQLqueriesthatproducewarningswillnotabortandreturnthewarnings
alongwiththequeryresults.TheoptioncanalsobeoverriddenforeachindividualAQLquery.
Enable/disableAQLquerytracking
--query.trackingflag
386
GeneralOptions
Iftrue,theserver'sAQLslowquerytrackingfeaturewillbeenabledbydefault.Trackingofqueriescanbedisabledbysettingtheoption
tofalse.
Thedefaultistrue.
Enable/disabletrackingofbindvariablesinAQLqueries
--query.tracking-with-bindvarsflag
Iftrue,thenthebindvariableswillbetrackedforallrunningandslowAQLqueries.Thisoptiononlyhasaneffectif --query.tracking
wassettotrue.Trackingofbindvariablescanbedisabledbysettingtheoptiontofalse.
Thedefaultistrue.
ThresholdforslowAQLqueries
--query.slow-thresholdvalue
BysettingvalueitcanbecontrolledafterwhatexecutiontimeanAQLqueryisconsidered"slow".Anyslowqueriesthatexceedthe
executiontimespecifiedinvaluewillbeloggedwhentheyarefinished.Thethresholdvalueisspecifiedinseconds.Trackingofslow
queriescanbeturnedoffentirelybysettingtheoption --query.trackingtofalse.
Thedefaultvalueis10.0.
Throwcollectionnotloadederror
--database.throw-collection-not-loaded-errorflag
Accessinganot-yetloadedcollectionwillautomaticallyloadacollectiononfirstaccess.Thisflagcontrolswhathappensincasean
operationwouldneedtowaitforanotherthreadtofinalizeloadingacollection.Ifsettotrue,thenthefirstoperationthataccessesan
unloadedcollectionwillloadit.Furtherthreadsthattrytoaccessthesamecollectionwhileitisstillloadingwillgetanerror(1238,
collectionnotloaded).Whentheinitialoperationhascompletedloadingthecollection,alloperationsonthecollectioncanbecarriedout
normally,anderror1238willnotbethrown.
Ifsettofalse,thefirstthreadthataccessesanot-yetloadedcollectionwillstillloadit.Otherthreadsthattrytoaccessthecollection
whileloadingwillnotfailwitherror1238butinsteadblockuntilthecollectionisfullyloaded.Thisconfigurationmightleadtoallserver
threadsbeingblockedbecausetheyareallwaitingforthesamecollectiontocompleteloading.Settingtheoptiontotruewillpreventthis
fromhappening,butrequiresclientstocatcherror1238andreactonit(maybebyschedulingaretryforlater).
Thedefaultvalueisfalse.
AQLQuerycachingmode
--query.cache-mode
TogglestheAQLquerycachebehavior.Possiblevaluesare:
off:donotusequerycache
on:alwaysusequerycache,exceptforqueriesthathavetheircacheattributesettofalse
demand:usequerycacheonlyforqueriesthathavetheircacheattributesettotrue
AQLQuerycachesize
--query.cache-entries
M aximumnumberofqueryresultsthatcanbestoredperdatabase-specificquerycache.Ifaqueryiseligibleforcachingandthenumber
ofitemsinthedatabase'squerycacheisequaltothisthresholdvalue,anothercachedqueryresultwillberemovedfromthecache.
Thisoptiononlyhasaneffectifthequerycachemodeissettoeitheronordemand.
JavaScriptcodeexecution
--javascript.allow-admin-execute
387
GeneralOptions
Thisoptioncanbeusedtocontrolwhetheruser-definedJavaScriptcodeisallowedtobeexecutedonserverbysendingviaHTTPtothe
APIendpoint /_admin/executewithanauthenticateduseraccount.Thedefaultvalueisfalse,whichdisablestheexecutionofuserdefinedcode.Thisisalsotherecommendedsettingforproduction.Intestenvironments,itmaybeconvenienttoturntheoptiononin
ordertosendarbitrarysetuporteardowncommandsforexecutionontheserver.
V8contexts
--javascript.v8-contextsnumber
SpecifiesthemaximumnumberofV8contextsthatarecreatedforexecutingJavaScriptcode.M orecontextsallowexecutingmore
JavaScriptactionsinparallel,providedthattherearealsoenoughthreadsavailable.PleasenotethateachV8contextwilluseasubstantial
amountofmemoryandrequiresperiodicCPUprocessingtimeforgarbagecollection.
NotethatthisvalueconfiguresthemaximumnumberofV8contextsthatcanbeusedinparallel.UponserverstartonlyasmanyV8
contextswillbecreatedasareconfiguredinoption --javascript.v8-contexts-minimum.TheactualnumberofavailableV8contextsmay
floatatruntimebetween --javascript.v8-contexts-minimumand --javascript.v8-contexts.WhenthereareunusedV8contextsthat
lingeraround,theserver'sgarbagecollectorthreadwillautomaticallydeletethem.
--javascript.v8-contexts-minimumnumber
SpecifiestheminimumnumberofV8contextsthatwillbepresentatanytimetheserverisrunning.TheactualnumberofV8contexts
willneverdropbelowthisvalue,butitmaygoupashighasspecifiedviatheoption --javascript.v8-contexts.
WhenthereareunusedV8contextsthatlingeraroundandthenumberofV8contextsisgreaterthan --javascript.v8-contexts-minimum
theserver'sgarbagecollectorthreadwillautomaticallydeletethem.
Garbagecollectionfrequency(time-based)
--javascript.gc-frequencyfrequency
Specifiesthefrequency(inseconds)fortheautomaticgarbagecollectionofJavaScriptobjects.Thissettingisusefultohavethegarbage
collectionstillworkinperiodswithnoorlittlenumbersofrequests.
Garbagecollectioninterval(request-based)
--javascript.gc-intervalinterval
Specifiestheinterval(approximatelyinnumberofrequests)thatthegarbagecollectionforJavaScriptobjectswillberunineachthread.
V8options
--javascript.v8-optionsoptions
OptionalargumentstopasstotheV8Javascriptengine.TheV8enginewillrunwithdefaultsettingsunlessexplicitoptionsarespecified
usingthisoption.TheoptionspassedwillbeforwardedtotheV8enginewhichwillparsethemonitsown.Passinginvalidoptionsmay
resultinanerrorbeingprintedonstderrandtheoptionbeingignored.
Optionsneedtobepassedinonestring,withV8optionnamesbeingprefixedwithdoubledashes.M ultipleoptionsneedtobeseparated
bywhitespace.TogetalistofallavailableV8options,youcanusethevalue"--help"asfollows:
--javascript.v8-options="--help"
AnotherexampleofspecificV8optionsbeingsetatstartup:
--javascript.v8-options="--log"
NamesandfeaturesorusableoptionsdependontheversionofV8beingused,andmightchangeinthefutureifadifferentversionofV8
isbeingusedinArangoDB.NotalloptionsofferedbyV8mightbesensibletouseinthecontextofArangoDB.Usethespecificoptions
onlyifyouaresurethattheyarenotharmfulfortheregulardatabaseoperation.
388
Write-AheadLogOptions
MMFilesWrite-aheadlogoptions
SinceArangoDB2.2,theM M Filesstorageenginewillwritealldata-modificationoperationsintoitswrite-aheadlog.
WithArangoDB3.2anotherStorageengineoptionbecomesavailable-RocksDB.IncaseofusingRocksDBmostofthesubsequent
optionsdon'thaveausefulmeaning.
Thewrite-aheadlogisasequenceoflogfilesthatarewritteninanappend-onlyfashion.Fulllogfileswilleventuallybegarbage-collected,
andtherelevantdatamightbetransferredintocollectionjournalsanddatafiles.Unneededandalreadygarbage-collectedlogfileswilleither
bedeletedorkeptforthepurposeofkeepingareplicationbacklog.
Directory
TheWALlogfilesdirectory: --wal.directory
Specifiesthedirectoryinwhichthewrite-aheadlogfilesshouldbestored.Ifthisoptionisnotspecified,itdefaultstothesubdirectory
journalsintheserver'sglobaldatabasedirectory.Ifthedirectoryisnotpresent,itwillbecreated.
Logfilesize
thesizeofeachWALlogfile --wal.logfile-sizeSpecifiesthefilesize(inbytes)foreachwrite-aheadlogfile.Thelogfilesizeshouldbe
chosensothateachlogfilecanstoreaconsiderableamountofdocuments.Thebiggerthelogfilesizeischosen,thelongeritwilltaketofill
upasinglelogfile,whichalsoinfluencesthedelayuntilthedatainalogfilewillbegarbage-collectedandwrittentocollectionjournalsand
datafiles.Italsoaffectshowlonglogfilerecoverywilltakeatserverstart.
Allowoversizeentries
whetherornotoversizeentriesareallowed --wal.allow-oversize-entriesWhetherornotitisallowedtostoreindividualdocuments
thatarebiggerthanwouldfitintoasinglelogfile.Settingtheoptiontofalsewillmakesuchoperationsfailwithanerror.Settingthe
optiontotruewillmakesuchoperationssucceed,butwithahighpotentialperformanceimpact.Thereasonisthatforeachoversize
operation,anindividualoversizelogfileneedstobecreatedwhichmayalsoblockotheroperations.Theoptionshouldbesettofalseifit
iscertainthatdocumentswillalwayshaveasizesmallerthanasinglelogfile.
Numberofreservelogfiles
maximumnumberofreservelogfiles --wal.reserve-logfilesThemaximumnumberofreservelogfilesthatArangoDBwillcreateina
backgroundprocess.Reservelogfilesareusefulinthesituationwhenanoperationneedstobewrittentoalogfilebutthereservespacein
thelogfileistoolowforstoringtheoperation.Inthiscase,anewlogfileneedstobecreatedtostoretheoperation.Creatingnewlogfiles
isnormallyslow,soArangoDBwilltrytopre-createlogfilesinabackgroundprocesssotherearealwaysreservelogfileswhentheactive
logfilegetsfull.ThenumberofreservelogfilesthatArangoDBkeepsinthebackgroundisconfigurablewiththisoption.
Numberofhistoriclogfiles
maximumnumberofhistoriclogfiles --wal.historic-logfilesThemaximumnumberofhistoriclogfilesthatArangoDBwillkeepafter
theyhavebeengarbage-collected.Ifnoreplicationisused,thereisnoneedtokeephistoriclogfilesexceptforhavingalocalchangelog.In
areplicationsetup,thenumberofhistoriclogfilesaffectstheamountofdataaslavecanfetchfromthemaster'slogs.Themorehistoric
logfiles,themorehistoricdataisavailableforaslave,whichisusefuliftheconnectionbetweenmasterandslaveisunstableorslow.Not
havingenoughhistoriclogfilesavailablemightleadtologfiledatabeingdeletedonthemasteralreadybeforeaslavehasfetchedit.
Syncinterval
intervalforautomatic,non-requesteddisksyncs --wal.sync-intervalTheinterval(inmilliseconds)thatArangoDBwilluseto
automaticallysynchronizedatainitswrite-aheadlogstodisk.Automaticsyncswillonlybeperformedfornot-yetsynchronizeddata,
andonlyforoperationsthathavebeenexecutedwithoutthewaitForSyncattribute.
Flushtimeout
389
Write-AheadLogOptions
WALflushtimeout`--wal.flush-timeoutThetimeout(inmilliseconds)thatArangoDBwillatmostwaitwhenflushingafullWALlogfile
todisk.Whenthetimeoutisreachedandtheflushisnotcompleted,theoperationthatrequestedtheflushwillfailwithalocktimeout
error.
Throttling
ThrottlewritestoWALwhenatleastsuchmanyoperationsarewaitingforgarbagecollection: --wal.throttle-when-pending
Themaximumvalueforthenumberofwrite-aheadloggarbage-collectionqueueelements.Ifsetto0,thequeuesizeisunbounded,andno
write-throttlingwilloccur.Ifsettoanon-zerovalue,write-throttlingwillautomaticallykickinwhenthegarbage-collectionqueue
containsatleastasmanyelementsasspecifiedbythisoption.Whilewrite-throttlingisactive,data-modificationoperationswill
intentionallybedelayedbyaconfigurableamountoftime.Thisistoensurethewrite-aheadloggarbagecollectorcancatchupwiththe
operationsexecuted.Write-throttlingwillstayactiveuntilthegarbage-collectionqueuesizegoesdownbelowthespecifiedvalue.Writethrottlingisturnedoffbydefault.
--wal.throttle-wait
Thisoptiondeterminesthemaximumwaittime(inmilliseconds)foroperationsthatarewrite-throttled.Ifwrite-throttlingisactiveanda
newwriteoperationistobeexecuted,itwillwaitforatmostthespecifiedamountoftimeforthewrite-aheadloggarbage-collection
queuesizetofallbelowthethrottlingthreshold.Ifthequeuesizedecreasesbeforethemaximumwaittimeisover,theoperationwillbe
executednormally.Ifthequeuesizedoesnotdecreasebeforethewaittimeisover,theoperationwillbeabortedwithanerror.This
optiononlyhasaneffectif --wal.throttle-when-pendinghasanon-zerovalue,whichisnotthedefault.
Numberofslots
M aximumnumberofslotstobeusedinparallel: --wal.slots
Configurestheamountofwriteslotsthewrite-aheadlogcangivetowriteoperationsinparallel.Anywriteoperationwillleaseaslotand
returnittothewrite-aheadlogwhenitisfinishedwritingthedata.Aslotwillremainblockeduntilthedatainitwassynchronizedto
disk.Afterthat,aslotbecomesreusablebyfollowingoperations.Therequirednumberofslotsisthusdeterminedbytheparallelityof
writeoperationsandthedisksynchronizationspeed.Slowdisksprobablyneedhighervalues,andfastdisksmayonlyrequireavalue
lowerthanthedefault.
Ignorelogfileerrors
Ignorelogfileerrorswhenopeninglogfiles: --wal.ignore-logfile-errors
Ignoresanyrecoveryerrorscausedbycorruptedlogfilesonstartup.Whensettofalse,therecoveryprocedureonstartupwillfailwithan
errorwheneveritencountersacorrupted(thatincludesonlyhalf-written)logfile.Thisisasecurityprecautiontopreventdatalossincase
ofdiskerrorsetc.Whentherecoveryprocedureabortsbecauseofcorruption,anycorruptedfilescanbeinspectedandfixed(orremoved)
manuallyandtheservercanberestartedafterwards.
Settingtheoptiontotruewillmaketheservercontinuewiththerecoveryprocedureevenincaseitdetectscorruptlogfileentries.Inthis
caseitwillstopatthefirstcorruptedlogfileentryandignoreallothers,whichmightcausedataloss.
Ignorerecoveryerrors
Ignorerecoveryerrors: --wal.ignore-recovery-errors
Ignoresanyrecoveryerrorsnotcausedbycorruptedlogfilesbutbylogicalerrors.Logicalerrorscanoccuriflogfilesoranyotherserver
datafileshavebeenmanuallyeditedortheserverissomehowmisconfigured.
Ignore(non-WAL)datafileerrors
Ignoredatafileerrorswhenloadingcollections: --database.ignore-datafile-errorsboolean
Ifsetto false,CRCmismatchandothererrorsincollectiondatafileswillleadtoacollectionnotbeingloadedatall.Thecollectionin
thiscasebecomesunavailable.IfsuchcollectionneedstobeloadedduringWALrecovery,theWALrecoverywillalsoabort(ifnotforced
withoption --wal.ignore-recovery-errorstrue).
390
Write-AheadLogOptions
Settingthisflagto falseprotectsusersfromunintentionallyusingacollectionwithcorrupteddatafiles,fromwhichonlyasubsetof
theoriginaldatacanberecovered.Workingwithsuchcollectioncouldleadtodatalossandfollowuperrors.Inordertoaccesssuch
collection,itisrequiredtoinspectandrepairthecollectiondatafilewiththedatafiledebugger(arango-dfdb).
Ifsetto true,CRCmismatchandothererrorsduringtheloadingofacollectionwillleadtothedatafilebeingpartiallyloaded,uptothe
positionofthefirsterror.Alldatauptountiltheinvalidpositionwillbeloaded.Thiswillenableuserstocontinuewithcollection
datafileseveniftheyarecorrupted,butthiswillresultinonlyapartialloadoftheoriginaldataandpotentialfollowuperrors.TheWAL
recoverywillstillabortwhenencounteringacollectionwithacorrupteddatafile,atleastif --wal.ignore-recovery-errorsisnotsetto
true.
Thedefaultvalueisfalse,socollectionswithcorrupteddatafileswillnotbeloadedatall,preventingpartialloadsandfollowuperrors.
However,ifsuchcollectionisrequiredatserverstartup,duringWALrecovery,theserverwillaborttherecoveryandrefusetostart.
391
ClusterOptions
ClustersOptions
Agencyendpoint
Listofagencyendpoints: --cluster.agency-endpointendpoint
Anagencyendpointtheservercanconnectto.Theoptioncanbespecifiedmultipletimes,sotheservercanuseaclusterofagency
servers.Endpointshavethefollowingpattern:
tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4
tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6
ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryption
ssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryption
AtleastoneendpointmustbespecifiedorArangoDBwillrefusetostart.Itisrecommendedtospecifyatleasttwoendpointsso
ArangoDBhasanalternativeendpointifoneofthembecomesunavailable.
Examples
--cluster.agency-endpointtcp://192.168.1.1:4001--cluster.agency-endpointtcp://192.168.1.2:4002...
Myaddress
Thisserver'saddress/endpoint: --cluster.my-addressendpoint
Theserver'sendpointforcluster-internalcommunication.Ifspecified,itmusthavethefollowingpattern:
tcp://ipv4-address:port-TCP/IPendpoint,usingIPv4
tcp://[ipv6-address]:port-TCP/IPendpoint,usingIPv6
ssl://ipv4-address:port-TCP/IPendpoint,usingIPv4,SSLencryption
ssl://[ipv6-address]:port-TCP/IPendpoint,usingIPv6,SSLencryption
Ifnoendpointisspecified,theserverwilllookupitsinternalendpointaddressintheagency.Ifnoendpointcanbefoundintheagency
fortheserver'sid,ArangoDBwillrefusetostart.
Examples
Listenonlyoninterfacewithaddress 192.168.1.1
--cluster.my-addresstcp://192.168.1.1:8530
Listenonallipv4andipv6addresses,whichareconfiguredonport 8530
--cluster.my-addressssl://[::]:8530
Myrole
Thisserver'srole: --cluster.my-role[dbserver|coordinator]
Theserver'srole.Isthisinstanceadbserver(backenddataserver)oracoordinator(frontendserverforexternalandapplicationaccess)
NodeID(deprecated)
Thisserver'sid: --cluster.my-local-infoinfo
Somelocalinformationabouttheserverinthecluster,thiscanforexamplebeanIPaddresswithaprocessIDoranystringuniquetothe
server.Specifyinginfoismandatoryonstartupiftheserverid(seebelow)isnotspecified.Eachserveroftheclustermusthaveaunique
localinfo.Thisisignoredifmy-idbelowisspecified.
392
ClusterOptions
Thisoptionisdeprecatedandwillberemovedinafuturerelease.Theclusternodeidshavebeendroppedinfavourofoncegenerated
UUIDs.
Moreadvancedoptions(shouldgenerallyremainuntouched)
Synchroneousreplicationtiming: --cluster.synchronous-replication-timeout-factordouble
Strechorclinchtimeoutsforinternalsynchroneousreplicationmechanismbetweendbservers.Allsuchtimeoutsareaffectedbythis
change.Pleasechangeonlywithintentandgreatcare.Defaultat 1.0.
Systemreplicationfactor: --cluster.system-replication-factorinteger
Changedefaultreplicationfactorforsystemcollections.Defaultat 2.
393
RocksDBEngineOptions
RocksDBengineoptions
RocksDBisahighlyconfigurablekey-valuestoreusedtopowerourRocksDBstorageengine.M ostoftheoptionsonthispagearepassthroughoptionstotheunderlyingRocksDBinstance,andwechangeveryfewoftheirdefaultsettings.
Dependingonthestorageengineyouhavechosentheavailabilityandthescopeoftheseoptionschanges.
Incaseyouhavechosen mmfilessomeofthefollowingoptionsapplytopersistentindexes.Incaseof rocksdbitwillapplytoall
datastoredaswellasindexes.
Pass-throughoptions
--rocksdb.wal-directory
AbsolutepathfortheRocksDBWALfiles.Ifleftempty,thiswilluseasubdirectory journalsinsidethedatadirectory.
Writebuffers
--rocksdb.write-buffer-size
Theamountofdatatobuildupineachin-memorybuffer(backedbyalogfile)beforeclosingthebufferandqueuingittobeflushedinto
standardstorage.Default:64M iB.Largervaluesmayimproveperformance,especiallyforbulkloads.
--rocksdb.max-write-buffer-number
Themaximumnumberofwritebuffersthatbuiltupinmemory.Ifthisnumberisreachedbeforethebufferscanbeflushed,writeswillbe
slowedorstalled.Default:2.
--rocksdb.min-write-buffer-number-to-merge
M inimumnumberofwritebuffersthatwillbemergedtogetherwhenflushingtonormalstorage.Default:1.
--rocksdb.max-total-wal-size
M aximumtotalsizeofWALfilesthat,whenreached,willforceaflushofallcolumnfamilieswhosedataisbackedbytheoldestWAL
files.Settingthistoalowvaluewilltriggerregularflushingofcolumnfamilydatafrommemtables,sothatWALfilescanbemovedtothe
archive.SettingthistoahighvaluewillavoidregularflushingbutmaypreventWALfilesfrombeingmovedtothearchiveandbeing
removed.
--rocksdb.delayed-write-rate(Hidden)
LimitedwriteratetoDB(inbytespersecond)ifwearewritingtothelastin-memorybufferallowedandweallowmorethan3buffers.
Default:16M iB/s.
LSMtreestructure
--rocksdb.num-levels
ThenumberoflevelsforthedatabaseintheLSM tree.Default:7.
--rocksdb.num-uncompressed-levels
Thenumberoflevelsthatdonotusecompression.Thedefaultvalueis2.LevelsabovethisnumberwilluseSnappycompressionto
reducethediskspacerequirementsforstoringdataintheselevels.
--rocksdb.dynamic-level-bytes
Iftrue,theamountofdataineachleveloftheLSM treeisdetermineddynamicallysoastominimizethespaceamplification;otherwise,
thelevelsizesarefixed.ThedynamicsizingallowsRocksDBtomaintainawell-structuredLSM treeregardlessoftotaldatasize.
Default:true.
--rocksdb.max-bytes-for-level-base
Themaximumtotaldatasizeinbytesinlevel-1oftheLSM tree.Onlyeffectiveif --rocksdb.dynamic-level-bytesisfalse.Default:
256M iB.
394
RocksDBEngineOptions
--rocksdb.max-bytes-for-level-multiplier
ThemaximumtotaldatasizeinbytesforlevelLoftheLSM treecanbecalculatedas max-bytes-for-level-base*(max-bytes-for-levelmultiplier^(L-1)).Onlyeffectiveif --rocksdb.dynamic-level-bytesisfalse.Default:10.
--rocksdb.level0-compaction-trigger
Compactionoflevel-0tolevel-1istriggeredwhenthismanyfilesexistinlevel-0.Settingthistoahighernumbermayhelpbulkwritesat
theexpenseofslowingdownreads.Default:2.
--rocksdb.level0-slowdown-trigger
Whenthismanyfilesaccumulateinlevel-0,writeswillbesloweddownto --rocksdb.delayed-write-ratetoallowcompactiontocatch
up.Default:20.
--rocksdb.level0-stop-trigger
Whenthismanyfilesaccumulateinlevel-0,writeswillbestoppedtoallowcompactiontocatchup.Default:36.
FileI/O
--rocksdb.compaction-read-ahead-size
Ifnon-zero,weperformbiggerreadswhendoingcompaction.Ifyou'rerunningRocksDBonspinningdisks,youshouldsetthistoat
least2M iB.ThatwayRocksDB'scompactionisdoingsequentialinsteadofrandomreads.Default:0.
--rocksdb.use-direct-reads(Hidden)
OnlymeaningfulonLinux.Ifset,use O_DIRECTforreadingfiles.Default:false.
--rocksdb.use-direct-io-for-flush-and-compaction(Hidden)
OnlymeaningfulonLinux.Ifset,use O_DIRECTforwritingfiles.Default:false.
--rocksdb.use-fsync(Hidden)
Ifset,issuean fsynccallwhenwritingtodisk(settofalsetoissue fdatasynconly.Default:false.
Backgroundtasks
--rocksdb.max-background-jobs
M aximumnumberofconcurrentbackgroundcompactionjobs,submittedtothelowprioritythreadpool.Default:numberofprocessors.
--rocksdb.num-threads-priority-high
Numberofthreadsforhighpriorityoperations(e.g.flush).Werecommendsettingthisequalto max-background-flushes.Default:
numberofprocessors/2.
--rocksdb.num-threads-priority-low
Numberofthreadsforlowpriorityoperations(e.g.compaction).Default:numberofprocessors/2.
Caching
--rocksdb.block-cache-size
Thisisthesizeoftheblockcacheinbytes.Increasingthismayimproveperformance.Ifthereislessthan4GiBofRAM onthesystem,
thedefaultvalueis256M iB.Ifthereismore,thedefaultis (systemRAMsize-2GiB)*0.3.
--rocksdb.block-cache-shard-bits
Thenumberofbitsusedtoshardtheblockcachetoallowconcurrentoperations.Tokeepindividualshardsatareasonablesize(i.e.at
least512KB),keepthisvaluetoatmost block-cache-shard-bits/512KB.Default: block-cache-size/2^19.
--rocksdb.table-block-size
Approximatesizeofuserdata(inbytes)packedperblockforuncompresseddata.
--rocksdb.recycle-log-file-num(Hidden)
Numberoflogfilestokeeparoundforrecycling.Default:0.
395
RocksDBEngineOptions
Miscellaneous
--rocksdb.optimize-filters-for-hits(Hidden)
Thisflagspecifiesthattheimplementationshouldoptimizethefiltersmainlyforcaseswherekeysarefoundratherthanalsooptimize
forthecasewherekeysarenot.Thiswouldbeusedincaseswheretheapplicationknowsthatthereareveryfewmissesorthe
performanceinthecaseofmissesisnotasimportant.Default:false.
--rocksdb.wal-recovery-skip-corrupted(Hidden)
Iftrue,skipcorruptedrecordsinWALrecovery.Default:false.
Non-Pass-ThroughOptions
--rocksdb.wal-file-timeout(Hidden)
TimeoutafterwhichunusedWALfilesaredeleted(inseconds).Default:10.0s.
DataofongoingtransactionsisstoredinRAM .Transactionsthatgettoobig(intermsofnumberofoperationsinvolvedorthetotalsize
ofdatacreatedormodifiedbythetransaction)willbecommittedautomatically.Effectivelythismeansthatbigusertransactionsaresplit
intomultiplesmallerRocksDBtransactionsthatarecommittedindividually.TheentireusertransactionwillnotnecessarilyhaveACID
propertiesinthiscase.
ThefollowingoptionscanbeusedtocontroltheRAM usageandautomaticintermediatecommitsfortheRocksDBengine:
--rocksdb.max-transaction-size
Transactionsizelimit(inbytes).TransactionsstoreallkeysandvaluesinRAM ,solargetransactionsruntheriskofcausingout-ofmemorysitations.Thissettingallowsyoutoensurethatdoesnothappenbylimitingthesizeofanyindividualtransaction.Transactions
whoseoperationswouldconsumemoreRAM thanthisthresholdvaluewillabortautomaticallywitherror32("resourcelimitexceeded").
--rocksdb.intermediate-commit-size
Ifthesizeofalloperationsinatransactionreachesthisthreshold,thetransactioniscommittedautomaticallyandanewtransactionis
started.Thevalueisspecifiedinbytes.
--rocksdb.intermediate-commit-count
Ifthenumberofoperationsinatransactionreachesthisvalue,thetransactioniscommittedautomaticallyandanewtransactionis
started.
396
HashCacheOptions
Hashcacheoptions
SinceArangoDB3.2,theseveralcorecomponentsoftheserveruseacachesystemwhichpoolsmemoryacrossmanydifferentcache
tables.Inordertoprovideintelligentinternalmemorymanagement,thesystemperiodicallyreclaimsmemoryfromcacheswhichareused
lessoftenandreallocatesittocacheswhichgetmoreactivity.
Cachesize
Globalsizelimitforallhashcaches: --cache.size
Theglobalcachingsystem,allcaches,andallthedatacontainedthereinwillfitinsidethislimit.Thesizeisspecifiedinbytes.Ifthereis
lessthan4GiBofRAM onthesystem,thedefaultvalueis256M iB.Ifthereismore,thedefaultis (systemRAMsize-2GiB)*0.3.
Rebalancinginterval
Timebetweencacherebalancingattempts: --cache.rebalancing-interval
Thevalueisspecifiedinmicrosecondswithadefaultof2secondsandaminimumof500milliseconds.
397
AsynchronousTasks
AsynchronousTasks
maximalqueuesize
M aximumsizeofthequeueforrequests: --server.maximal-queue-sizesize
Specifiesthemaximumsizeofthequeueforasynchronoustaskexecution.Ifthequeuealreadycontainssizetasks,newtaskswillbe
rejecteduntilothertasksarepoppedfromthequeue.Settingthisvaluemayhelppreventingfromrunningoutofmemoryifthequeueis
filledupfasterthantheservercanprocessrequests.
398
Durability
DurabilityConfiguration
GlobalConfiguration
Thereareglobalconfigurationvaluesfordurability,whichcanbeadjustedbyspecifyingthefollowingconfigurationoptions:
defaultwaitforsyncbehavior --database.wait-for-syncbooleanDefaultwait-for-syncvalue.Canbeoverwrittenwhencreatinganew
collection.Thedefaultisfalse.
forcesyncingofcollectionpropertiestodisk --database.force-sync-propertiesbooleanForcesyncingofcollectionpropertiestodisk
aftercreatingacollectionorupdatingitsproperties.Ifturnedoff,nofsyncwillhappenforthecollectionanddatabasepropertiesstored
in parameter.jsonfilesinthefilesystem.Turningoffthisoptionwillspeedupworkloadsthatcreateanddropalotofcollections(e.g.
testsuites).Thedefaultistrue.
intervalforautomatic,non-requesteddisksyncs --wal.sync-intervalTheinterval(inmilliseconds)thatArangoDBwilluseto
automaticallysynchronizedatainitswrite-aheadlogstodisk.Automaticsyncswillonlybeperformedfornot-yetsynchronizeddata,
andonlyforoperationsthathavebeenexecutedwithoutthewaitForSyncattribute.
Per-collectionconfiguration
Youcanalsoconfigurethedurabilitybehavioronaper-collectionbasis.UsetheArangoDBshelltochangetheseproperties.
getsorsetsthepropertiesofacollection collection.properties()Returnsanobjectcontainingallcollectionproperties.
waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheM M Filesstorageengineonly.
isVolatile:IftruethenthecollectiondatawillbekeptinmemoryonlyandArangoDBwillnotwriteorsyncthedatatodisk.This
optionismeaningfulfortheM M Filesstorageengineonly.
keyOptions(optional)additionaloptionsforkeygeneration.ThisisaJSONarraycontainingthefollowingattributes(note:someof
theattributesareoptional):
type:thetypeofthekeygeneratorusedforthecollection.
allowUserKeys:ifsettotrue,thenitisallowedtosupplyownkeyvaluesinthe_keyattributeofadocument.Ifsettofalse,
thenthekeygeneratorwillsolelyberesponsibleforgeneratingkeysandsupplyingownkeyvaluesinthe_keyattributeof
documentsisconsideredanerror.
increment:incrementvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
offset:initialoffsetvalueforautoincrementkeygenerator.Notusedforotherkeygeneratortypes.
indexBuckets:numberofbucketsintowhichindexesusingahashtablearesplit.Thedefaultis16andthisnumberhastobeapower
of2andlessthanorequalto1024.ThisoptionismeaningfulfortheM M Filesstorageengineonly.Forverylargecollectionsone
shouldincreasethistoavoidlongpauseswhenthehashtablehastobeinitiallybuiltorresized,sincebucketsareresized
individuallyandcanbeinitiallybuiltinparallel.Forexample,64mightbeasensiblevalueforacollectionwith100000000
documents.Currently,onlytheedgeindexrespectsthisvalue,butotherindextypesmightfollowinfutureArangoDBversions.
Changes(seebelow)areappliedwhenthecollectionisloadedthenexttime.Inaclustersetup,theresultwillalsocontainthe
followingattributes:
numberOfShards:thenumberofshardsofthecollection.
shardKeys:containsthenamesofdocumentattributesthatareusedtodeterminethetargetshardfordocuments.
collection.properties(properties)Changesthecollectionproperties.propertiesmustbeaobjectwithoneormoreofthe
followingattribute(s):
waitForSync:Iftruecreatingadocumentwillonlyreturnafterthedatawassyncedtodisk.
journalSize:Thesizeofthejournalinbytes.ThisoptionismeaningfulfortheM M Filesstorageengineonly.
indexBuckets:Seeabove,changesareonlyappliedwhenthecollectionisloadedthenexttime.Thisoptionismeaningfulforthe
M M Filesstorageengineonly.Note:itisnotpossibletochangethejournalsizeafterthejournalordatafilehasbeencreated.
Changingthisparameterwillonlyeffectnewlycreatedjournals.Alsonotethatyoucannotlowerthejournalsizetolessthensizeof
thelargestdocumentalreadystoredinthecollection.Note:someothercollectionproperties,suchastype,isVolatile,orkeyOptions
cannotbechangedoncethecollectioniscreated.
Examples
399
Durability
Readallproperties
arangosh>db.example.properties();
showexecutionresults
Changeaproperty
arangosh>db.example.properties({waitForSync:true});
showexecutionresults
Per-operationconfiguration
M anydata-modificationoperationsandalsoArangoDB'stransactionsallowtospecifyawaitForSyncattribute,whichwhensetensures
theoperationdatahasbeensynchronizedtodiskwhentheoperationreturns.
Disk-UsageConfiguration
TheamountofdiskspaceusedbyArangoDBisdeterminedbyafewconfigurationoptions.
GlobalConfiguration
ThetotalamountofdiskstoragerequiredbyArangoDBisdeterminedbythesizeofthewrite-aheadlogfilesplusthesizesofthe
collectionjournalsanddatafiles.
Therearethefollowingoptionsforconfiguringthenumberandsizesofthewrite-aheadlogfiles:
maximumnumberofreservelogfiles --wal.reserve-logfilesThemaximumnumberofreservelogfilesthatArangoDBwillcreateina
backgroundprocess.Reservelogfilesareusefulinthesituationwhenanoperationneedstobewrittentoalogfilebutthereservespacein
thelogfileistoolowforstoringtheoperation.Inthiscase,anewlogfileneedstobecreatedtostoretheoperation.Creatingnewlogfiles
isnormallyslow,soArangoDBwilltrytopre-createlogfilesinabackgroundprocesssotherearealwaysreservelogfileswhentheactive
logfilegetsfull.ThenumberofreservelogfilesthatArangoDBkeepsinthebackgroundisconfigurablewiththisoption.
maximumnumberofhistoriclogfiles --wal.historic-logfilesThemaximumnumberofhistoriclogfilesthatArangoDBwillkeepafter
theyhavebeengarbage-collected.Ifnoreplicationisused,thereisnoneedtokeephistoriclogfilesexceptforhavingalocalchangelog.In
areplicationsetup,thenumberofhistoriclogfilesaffectstheamountofdataaslavecanfetchfromthemaster'slogs.Themorehistoric
logfiles,themorehistoricdataisavailableforaslave,whichisusefuliftheconnectionbetweenmasterandslaveisunstableorslow.Not
havingenoughhistoriclogfilesavailablemightleadtologfiledatabeingdeletedonthemasteralreadybeforeaslavehasfetchedit.
thesizeofeachWALlogfile --wal.logfile-sizeSpecifiesthefilesize(inbytes)foreachwrite-aheadlogfile.Thelogfilesizeshouldbe
chosensothateachlogfilecanstoreaconsiderableamountofdocuments.Thebiggerthelogfilesizeischosen,thelongeritwilltaketofill
upasinglelogfile,whichalsoinfluencesthedelayuntilthedatainalogfilewillbegarbage-collectedandwrittentocollectionjournalsand
datafiles.Italsoaffectshowlonglogfilerecoverywilltakeatserverstart.
whetherornotoversizeentriesareallowed --wal.allow-oversize-entriesWhetherornotitisallowedtostoreindividualdocuments
thatarebiggerthanwouldfitintoasinglelogfile.Settingtheoptiontofalsewillmakesuchoperationsfailwithanerror.Settingthe
optiontotruewillmakesuchoperationssucceed,butwithahighpotentialperformanceimpact.Thereasonisthatforeachoversize
operation,anindividualoversizelogfileneedstobecreatedwhichmayalsoblockotheroperations.Theoptionshouldbesettofalseifit
iscertainthatdocumentswillalwayshaveasizesmallerthanasinglelogfile.Whendatagetscopiedfromthewrite-aheadlogfilesintothe
journalsordatafilesofcollections,fileswillbecreatedonthecollectionlevel.Howbigthesefilesareisdeterminedbythefollowing
globalconfigurationvalue:
--database.maximal-journal-sizesizeM aximalsizeofjournalinbytes.Canbeoverwrittenwhencreatinganewcollection.Notethat
thisalsolimitsthemaximalsizeofasingledocument.Thedefaultis32MB.
400
Durability
Per-collectionconfiguration
Thejournalsizecanalsobeadjustedonaper-collectionlevelusingthecollection'spropertiesmethod.
401
Encryption
Encryption
ThisfeatureisonlyavailableintheEnterpriseEdition.
WhenyoustoresensitivedatainyourArangoDBdatabase,youwanttoprotectthatdataunderallcircumstances.Atruntimeyouwill
protectitwithSSLtransportencryptionandstrongauthentication,butwhenthedataisalreadyondisk,youalsoneedprotection.That
iswheretheEncryptionfeaturecomesin.
TheEncryptionfeatureofArangoDBwillencryptalldatathatArangoDBisstoringinyourdatabasebeforeitiswrittentodisk.
ThedataisencryptedwithAES-256-CTR,whichisastrongencryptionalgorithm,thatisverysuitableformulti-processor
environments.Thismeansthatyourdataissafe,butyourdatabaseisstillfast,evenunderload.
M ostmodernCPU'shavebuiltinsupportforhardwareAESencryption,whichmakesitevenfaster.
Note:TheEncryptionfeaturerequirestheRocksDBstorageengine.
Encryptionkeys
TheEncryptionfeatureofArangoDBrequiresasingle32-bytekeyperserver.Itisrecommendedtouseadifferentkeyforeachserver
(whenoperatinginaclusterconfiguration).M akesuretoprotectthesekeys!
Thatmeans:
Donotwritethemtopersistentdisksoryourserver(s),alwaysstorethemonanin-memory( tmpfs)filesystem.
Transportyourkeyssafelytoyourserver(s).Therearevarioustoolsformanagingsecretslikethis(e.g.vaultproject.io).
Storeacopyofyourkeyofflineinasafeplace.Ifyouloseyourkey,thereisNOwaytogetyourdataback.
Configuration
Toactivateencryptionofyourdatabase,passthefollowingoptionto arangod.
$arangod\
--rocksdb.encryption-keyfile=/mytmpfs/mySecretKey\
--server.storage-engine=rocksdb
Note:Youalsohavetoactivatethe rocksdbstorageengine.
M akesuretopassthisoptiontheveryfirsttimeyoustartyourdatabase.Youcannotencryptadatabasethatalreadyexists.
Creatingkeys
Theencryptionkeyfilemustcontain32bytesofrandomdata.
Youcancreateitwithacommandlinethis.
ddif=/dev/randombs=1count=32of=yourSecretKeyFile
Forsecurity,itisbesttocreatethesekeysoffline(awayfromyourdatabaseservers)anddirectlystoretheminyousecretmanagement
tool.
402
Auditing
Auditing
ThisfeatureisavailableintheEnterpriseEdition.
Auditingallowsyoutomonitoraccesstothedatabaseindetail.Ingeneralauditlogsareoftheform
2016-01-0112:00:00|server|username|database|client-ip|authentication|text1|text2|...
Thetime-stampisinGM T.Thisallowstoeasilymatchlogentriesfromserversindifferenttimezones.
Thenameoftheserver.Youcanspecifyacustomnameonstartup.Otherwisethedefaulthostnameisused.
Theusernameisthe(authenticatedorunauthenticated)namesuppliedbytheclient.Adash -isprintedifnonamewasgivenbythe
client.
Thedatabasedescribesthedatabasethatwasaccessed.Pleasenotethattherearenodatabasecrossingqueries.Eachaccessisrestrictedto
onedatabase.
Theclient-ipdescribesthesourceoftherequest.
Theauthenticationdetailsthemethodsusedtoauthenticatetheuser.
Detailsabouttherequestsfollowintheadditionalfields.
403
Configuration
AuditConfiguration
ThisfeatureisavailableintheEnterpriseEdition.
Output
--audit.outputoutput
Specifiesthetargetoftheauditlog.Possiblevaluesare
file://filenamewherefilenamecanberelativeorabsolute.
syslog://facilityor syslog://facility/application-nametologintoasyslogserver.
Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputformultipletargets.
Hostname
--audit.hostnamename
Thenameoftheserverusedinauditlogmessages.Bydefaultthesystemhostnameisused.
404
Events
AuditEvents
ThisfeatureisavailableintheEnterpriseEdition.
Authentication
Unknownauthenticationmethods
2016-10-0315:44:23|server1|-|database1|127.0.0.1:61525|-|unknownauthenticationmethod|/_api/version
Missingcredentials
2016-10-0315:39:49|server1|-|database1|127.0.0.1:61498|-|credentialsmissing|/_api/version
Wrongcredentials
2016-10-0315:47:26|server1|user1|database1|127.0.0.1:61528|httpbasic|credentialswrong|/_api/version
Passwordchangerequired
2016-10-0316:18:53|server1|user1|database1|127.0.0.1:62257|-|passwordchangerequired|/_api/version
JWTloginsucceeded
2016-10-0317:21:22|server1|-|database1|127.0.0.1:64214|httpjwt|user'root'authenticated|/_open/auth
Pleasenote,thattheusergivenasthirdpartistheuserthatrequestedthelogin.Ingeneral,itwillbeempty.
JWTloginfailed
2016-10-0317:21:22|server1|-|database1|127.0.0.1:64214|httpjwt|user'root'wrongcredentials|/_open/auth
Pleasenote,thattheusergivenasthirdpartistheuserthatrequestedthelogin.Ingeneral,itwillbeempty.
Authorization
Usernotauthorizedtoaccessdatabase
2016-10-0316:20:52|server1|user1|database2|127.0.0.1:62262|httpbasic|notauthorized|/_api/version
Databases
Createadatabase
2016-10-0415:33:25|server1|user1|database1|127.0.0.1:56920|httpbasic|createdatabase'database1'|ok|/_api/dat
abase
405
Events
Dropadatabase
2016-10-0415:33:25|server1|user1|database1|127.0.0.1:56920|httpbasic|deletedatabase'database1'|ok|/_api/dat
abase
Collections
Createacollection
2016-10-0517:35:57|server1|user1|database1|127.0.0.1:51294|httpbasic|createcollection'collection1'|ok|/_api
/collection
Truncateacollection
2016-10-0517:36:08|server1|user1|database1|127.0.0.1:51294|httpbasic|truncatecollection'collection1'|ok|/_a
pi/collection/collection1/truncate
Dropacollection
2016-10-0517:36:30|server1|user1|database1|127.0.0.1:51294|httpbasic|deletecollection'collection1'|ok|/_api
/collection/collection1
Indexes
Createaindex
2016-10-0518:19:40|server1|user1|database1|127.0.0.1:52467|httpbasic|createindexin'collection1'|ok|{"field
s":["a"],"sparse":false,"type":"skiplist","unique":false}|/_api/index?collection=collection1
Dropaindex
2016-10-0518:18:28|server1|user1|database1|127.0.0.1:52464|httpbasic|dropindex':44051'|ok|/_api/index/colle
ction1/44051
Documents
Readingasingledocument
2016-10-0412:27:55|server1|user1|database1|127.0.0.1:53699|httpbasic|createdocumentok|/_api/document/collecti
on1
Replacingasingledocument
2016-10-0412:28:08|server1|user1|database1|127.0.0.1:53699|httpbasic|replacedocumentok|/_api/document/collect
ion1/21456?ignoreRevs=false
Modifyingasingledocument
2016-10-0412:28:15|server1|user1|database1|127.0.0.1:53699|httpbasic|modifydocumentok|/_api/document/collecti
on1/21456?keepNull=true&ignoreRevs=false
406
Events
Deletingasingledocument
2016-10-0412:28:23|server1|user1|database1|127.0.0.1:53699|httpbasic|deletedocumentok|/_api/document/collecti
on1/21456?ignoreRevs=false
Forexample,ifsomeonestriestodeleteanon-existingdocument,itwillbeloggedas
2016-10-0412:28:26|server1|user1|database1|127.0.0.1:53699|httpbasic|deletedocumentfailed|/_api/document/coll
ection1/21456?ignoreRevs=false
Queries
2016-10-0612:12:10|server1|user1|database1|127.0.0.1:54232|httpbasic|querydocument|ok|foriincollection1r
eturni|/_api/cursor
407
Replication
IntroductiontoReplication
Replicationallowsyoutoreplicatedataontoanothermachine.ItformsthebaseofalldisasterrecoveryandfailoverfeaturesArangoDB
offers.
ArangoDBoffersasynchronousandsynchronousreplicationwhichbothhavetheirprosandcons.Bothmodesmayandshouldbe
combinedinarealworldscenarioandbeappliedintheusecasewheretheyexcelmost.
Wewilldescribeprosandconsofeachoftheminthefollowingsections.
Synchronousreplication
Synchronousreplicationonlyworkswithinaclusterandistypicallyusedformissioncriticaldatawhichmustbeaccessibleatalltimes.
Synchronousreplicationgenerallystoresacopyofashard'sdataonanotherdbserverandkeepsitinsync.Essentially,whenstoringdata
afterenablingsynchronousreplicationtheclusterwillwaitforallreplicastowriteallthedatabeforegreenlightingthewriteoperationto
theclient.Thiswillnaturallyincreasethelatencyabit,sinceonemorenetworkhopisneededforeachwrite.However,itwillenablethe
clustertoimmediatelyfailovertoareplicawheneveranoutagehasbeendetected,withoutlosinganycommitteddata,andmostly
withoutevensignalinganerrorconditiontotheclient.
Synchronousreplicationisorganizedsuchthateveryshardhasaleaderand r-1followers,where rdenotedthereplicationfactor.
Thenumberoffollowerscanbecontrolledusingthe replicationFactorparameterwheneveryoucreateacollection,the
replicationFactorparameteristhetotalnumberofcopiesbeingkept,thatis,itisoneplusthenumberoffollowers.
Satellitecollections
SatellitecollectionsaresynchronouslyreplicatedcollectionshavingadynamicreplicationFactor.Theywillreplicatealldatatoall
databaseserversallowingthedatabaseserverstojoindatalocallyinsteadofdoingheavynetworkoperations.
Satellitecollectionsareanenterpriseonlyfeature.
Asynchronousreplication
InArangoDBanywriteoperationwillbeloggedtothewrite-aheadlog.WhenusingAsynchronousreplicationslaveswillconnecttoa
masterandapplyalltheeventsfromtheloginthesameorderlocally.Afterthat,theywillhavethesamestateofdataasthemaster
database.
408
AsynchronousReplication
Asynchronousreplication
Asynchronousreplicationworksbyloggingeverydatamodificationonamasterandreplayingtheseeventsonanumberofslaves.
Transactionsarehonoredinreplication,i.e.transactionalwriteoperationswillbecomevisibleonslavesatomically.
Asallwriteoperationswillbeloggedtoamasterdatabase'swrite-aheadlog,thereplicationinArangoDBcurrentlycannotbeusedfor
write-scaling.ThemainpurposesofthereplicationincurrentArangoDBaretoprovideread-scalabilityand"hotbackups"ofspecific
databases.
Itispossibletoconnectmultipleslavedatabasestothesamemasterdatabase.Slavedatabasesshouldbeusedasread-onlyinstances,and
nouser-initiatedwriteoperationsshouldbecarriedoutonthem.Otherwisedataconflictsmayoccurthatcannotbesolvedautomatically,
andthatwillmakethereplicationstop.
Inanasynchronousreplicationscenarioslaveswillpullchangesfromthemasterdatabase.Slavesneedtoknowtowhichmasterdatabase
theyshouldconnectto,butamasterdatabaseisnotawareoftheslavesthatreplicatefromit.Whenthenetworkconnectionbetweenthe
masterdatabaseandaslavegoesdown,writeoperationsonthemastercancontinuenormally.Whenthenetworkisupagain,slavescan
reconnecttothemasterdatabaseandtransfertheremainingchanges.Thiswillhappenautomaticallyprovidedslavesareconfigured
appropriately.
Replicationlag
Inthissetup,writeoperationsareappliedfirstinthemasterdatabase,andappliedintheslavedatabase(s)afterwards.
Forexample,let'sassumeawriteoperationisexecutedinthemasterdatabaseatpointintimet0.Tomakeaslavedatabaseapplythe
sameoperation,itmustfirstfetchthewriteoperation'sdatafrommasterdatabase'swrite-aheadlog,thenparseitandapplyitlocally.
Thiswillhappenatsomepointintimeaftert0,let'ssayt1.
Thedifferencebetweent1andt0iscalledthereplicationlag,anditisunavoidableinasynchronousreplication.Theamountofreplication
lagdependsonmanyfactors,afewofwhichare:
thenetworkcapacitybetweentheslavesandthemaster
theloadofthemasterandtheslaves
thefrequencyinwhichslavespollthemasterforupdates
Betweent0andt1,thestateofdataonthemasterisnewerthanthestateofdataontheslave(s).Atpointintimet1,thestateofdataon
themasterandslave(s)isconsistentagain(providednonewdatamodificationshappenedonthemasterinbetween).Thus,the
replicationwillleadtoaneventuallyconsistentstateofdata.
Replicationconfiguration
Thereplicationisturnedoffbydefault.Inordertocreateamaster-slavesetup,theso-calledreplicationapplierneedstobeenabledon
theslavedatabases.
Replicationisconfiguredonaper-databaselevel.Ifmultipledatabasearetobereplicated,thereplicationmustbesetupindividuallyper
database.
Thereplicationapplierontheslavecanbeusedtoperformaone-timesynchronizationwiththemaster(andthenstop),ortoperforman
ongoingreplicationofchanges.Toresumereplicationonslaverestart,theautoStartattributeofthereplicationappliermustbesettotrue.
Replicationoverhead
Asthemasterserversarelogginganywriteoperationinthewrite-ahead-loganywayreplicationdoesn'tcauseanyextraoverheadonthe
master.Howeveritwillofcoursecausesomeoverheadforthemastertoserveincomingreadrequestsoftheslaves.Returningthe
requesteddataishoweveratrivialtaskforthemasterandshouldnotresultinanotableperformancedegrationinproduction.
409
AsynchronousReplication
Components
ReplicationLogger
Purpose
Thereplicationloggerwillwritealldata-modificationoperationsintothewrite-aheadlog.Thislogmaythenbereadbyclientstoreplay
anydatamodificationonadifferentserver.
Checkingthestate
Toquerythecurrentstateofthelogger,usethestatecommand:
require("@arangodb/replication").logger.state();
Theresultmightlooklikethis:
{
"state":{
"running":true,
"lastLogTick":"133322013",
"totalEvents":16,
"time":"2014-07-06T12:58:11Z"
},
"server":{
"version":"2.2.0-devel",
"serverId":"40897075811372"
},
"clients":{
}
}
Therunningattributewillalwaysbetrue.InearlierversionsofArangoDBthereplicationwasoptionalandthiscouldhavebeenfalse.
ThetotalEventsattributeindicateshowmanylogeventshavebeenloggedsincethestartoftheArangoDBserver.Finally,thelastLogTick
valueindicatestheidofthelastoperationthatwaswrittentotheserver'swrite-aheadlog.Itcanbeusedtodeterminewhethernew
operationswerelogged,andisalsousedbythereplicationapplierforincrementalfetchingofdata.
Note:ThereplicationloggerstatecanalsobequeriedviatheHTTPAPI.
Toquerywhichdatarangesarestillavailableforreplicationclientstofetch,theloggerprovidesthefirstTickandtickRangesfunctions:
require("@arangodb/replication").logger.firstTick();
ThiswillreturntheminimumtickvaluethattheservercanprovidetoreplicationclientsviaitsreplicationAPIs.ThetickRangesfunction
returnstheminimumandmaximumtickvaluesperlogfile:
require("@arangodb/replication").logger.tickRanges();
ReplicationApplier
Purpose
Thepurposeofthereplicationapplieristoreaddatafromamasterdatabase'seventlog,andapplythemlocally.Theapplierwillcheck
themasterdatabasefornewoperationsperiodically.Itwillperformanincrementalsynchronization,i.e.onlyaskingthemasterfor
operationsthatoccurredafterthelastsynchronization.
410
AsynchronousReplication
Thereplicationapplierdoesnotgetnotifiedbythemasterdatabasewhenthereare"new"operationsavailable,butinsteadusesthepull
principle.Itmightthustakesometime(theso-calledreplicationlag)beforeanoperationfromthemasterdatabasegetsshippedtoand
appliedinaslavedatabase.
Thereplicationapplierofadatabaseisruninaseparatethread.Itmayencounterproblemswhenanoperationfromthemastercannotbe
appliedsafely,orwhentheconnectiontothemasterdatabasegoesdown(networkoutage,masterdatabaseisdownorunavailableetc.).
Inthiscase,thedatabase'sreplicationapplierthreadmightterminateitself.Itisthenuptotheadministratortofixtheproblemand
restartthedatabase'sreplicationapplier.
Ifthereplicationappliercannotconnecttothemasterdatabase,orthecommunicationfailsatsomepointduringthesynchronization,the
replicationapplierwilltrytoreconnecttothemasterdatabase.Itwillgiveupreconnectingonlyafteraconfigurableamountof
connectionattempts.
Thereplicationapplierstateisqueryableatanytimebyusingthestatecommandoftheapplier.Thiswillreturnthestateoftheapplier
ofthecurrentdatabase:
require("@arangodb/replication").applier.state();
Theresultmightlooklikethis:
{
"state":{
"running":true,
"lastAppliedContinuousTick":"152786205",
"lastProcessedContinuousTick":"152786205",
"lastAvailableContinuousTick":"152786205",
"progress":{
"time":"2014-07-06T13:04:57Z",
"message":"fetchingmasterlogfromoffset152786205",
"failedConnects":0
},
"totalRequests":38,
"totalFailedConnects":0,
"totalEvents":1,
"lastError":{
"errorNum":0
},
"time":"2014-07-06T13:04:57Z"
},
"server":{
"version":"2.2.0-devel",
"serverId":"210189384542896"
},
"endpoint":"tcp://master.example.org:8529",
"database":"_system"
}
Therunningattributeindicateswhetherthereplicationapplierofthecurrentdatabaseiscurrentlyrunningandpollingtheserverat
endpointfornewevents.
Theprogress.failedConnectsattributeshowshowmanyfailedconnectionattemptsthereplicationappliercurrentlyhasencounteredina
row.Incontrast,thetotalFailedConnectsattributeindicateshowmanyfailedconnectionattemptstheapplierhasmadeintotal.The
totalRequestsattributeshowshowmanyrequeststheapplierhassenttothemasterdatabaseintotal.ThetotalEventsattributeshows
howmanylogeventstheapplierhasreadfromthemaster.
Theprogress.messagesub-attributeprovidesabriefhintofwhattheappliercurrentlydoes(ifitisrunning).ThelastErrorattributealso
hasanoptionalerrorMessagesub-attribute,showingthelatesterrormessage.TheerrorNumsub-attributeofthelastErrorattributecan
beusedbyclientstoprogrammaticallycheckforerrors.Itshouldbe0ifthereisnoerror,anditshouldbenon-zeroiftheapplier
terminateditselfduetoaproblem.
Hereisanexampleofthestateafterthereplicationapplierterminateditselfdueto(repeated)connectionproblems:
{
"state":{
"running":false,
"progress":{
"time":"2014-07-06T13:14:37Z",
411
AsynchronousReplication
"message":"applierstopped",
"failedConnects":6
},
"totalRequests":79,
"totalFailedConnects":11,
"totalEvents":0,
"lastError":{
"time":"2014-07-06T13:09:41Z",
"errorMessage":"couldnotconnecttomasterattcp://master.example.org:8529:Couldnotconnectto'tcp:/...",
"errorNum":1400
},
...
}
}
Note:thestateofadatabase'sreplicationapplierisqueryableviatheHTTPAPI,too.PleaserefertoHTTPInterfaceforReplicationfor
moredetails.
All-in-onesetup
Tocopytheinitialdatafromtheslavetothemasterandstartthecontinuousreplication,thereisanall-in-onecommandsetupReplication:
require("@arangodb/replication").setupReplication(configuration);
Thefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthe_systemdatabase.Notethatitshouldbe
runontheslaveandnotthemaster:
db._useDatabase("_system");
require("@arangodb/replication").setupReplication({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
verbose:false,
includeSystem:false,
incremental:true,
autoResync:true
});
Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationisstarted,orincasetheinitial
synchronizationhasfailed.
Iftheinitialsynchronizationissuccessful,thecommandwillstorethegivenconfigurationontheslave.Italsoconfiguresthecontinuous
replicationtostartautomaticallyiftheslaveisrestarted,i.e.autoStartissettotrue.
Ifthecommandisrunwhiletheslave'sreplicationapplierisalreadyrunning,itwillfirststoptherunningapplier,dropitsconfiguration
anddoaresynchronizationofdatawiththemaster.Itwillthenusetheprovidedconfigration,overwritinganypreviouslyexisting
replicationconfigurationontheslave.
StartingandStopping
Tomanuallystartandstoptheapplierinthecurrentdatabase,thestartandstopcommandscanbeusedlikethis:
require("@arangodb/replication").applier.start(<tick>);
require("@arangodb/replication").applier.stop();
Note:Startingareplicationapplierwithoutsettingupaninitialconfigurationwillfail.Thereplicationapplierwilllookforits
configurationinafilenamedREPLICATION-APPLIER-CONFIGinthecurrentdatabase'sdirectory.Ifthefileisnotpresent,ArangoDB
willusesomedefaultconfiguration,butitcannotguesstheendpoint(theaddressofthemasterdatabase)theappliershouldconnectto.
Thusstartingtheapplierwithoutconfigurationwillfail.
Notethatatthefirsttimeyoustarttheapplier,youshouldpassthevaluereturnedinthelastLogTickattributeoftheinitialsync
operation.
412
AsynchronousReplication
Note:Startingadatabase'sreplicationapplierviathestartcommandwillnotnecessarilystarttheapplieronthenextandfollowing
ArangoDBserverrestarts.Additionally,stoppingadatabase'sreplicationappliermanuallywillnotnecessarilypreventtheapplierfrom
beingstartedagainonthenextserverstart.Allofthisisconfigurableseparately(hangonreading).
Note:whenstoppingandrestartingthereplicationapplierofdatabase,itwillresumewhereitlaststopped.Thisissensiblebecause
replicationlogeventsshouldbeappliedincrementally.Ifthereplicationapplierofadatabasehasneverbeenstartedbefore,itneedssome
tickvaluefromthemaster'slogfromwhichtostartfetchingevents.
Thereisonecaveattoconsiderwhenstoppingareplicationontheslave:iftherearestillongoingreplicatedtransactionsthatareneither
committedoraborted,stoppingthereplicationapplierwillcausetheseoperationstobelostfortheslave.Ifthesetransactionscommiton
themasterlaterandthereplicationisresumed,theslavewillnotbeabletocommitthesetransactions,too.Thusstoppingthereplication
applierontheslavemanuallyshouldonlybedoneifthereiscertaintythattherearenoongoingtransactionsonthemaster.
Configuration
Toconfigurethereplicationapplierofaspecificdatabase,usethepropertiescommand.Usingitwithoutanyargumentswillreturnthe
applier'scurrentconfiguration:
require("@arangodb/replication").applier.properties();
Theresultmightlooklikethis:
{
"requestTimeout":600,
"connectTimeout":10,
"ignoreErrors":0,
"maxConnectRetries":10,
"chunkSize":0,
"autoStart":false,
"adaptivePolling":true,
"includeSystem":true,
"requireFromPresent":false,
"autoResync":false,
"autoResyncRetries":2,
"verbose":false
}
Note:Thereisnoendpointattributeconfiguredyet.Theendpointattributeisrequiredforthereplicationappliertobestartable.Youmay
alsowanttoconfigureausernameandpasswordfortheconnectionviatheusernameandpasswordattributes.
require("@arangodb/replication").applier.properties({
endpoint:"tcp://master.domain.org:8529",
username:"root",
password:"secret",
verbose:false
});
Thiswillre-configurethereplicationapplierforthecurrentdatabase.Theconfigurationwillbeusedfromthenextstartofthereplication
applier.Thereplicationappliercannotbere-configuredwhileitisrunning.Itmustbestoppedfirsttobere-configured.
TomakethereplicationapplierofthecurrentdatabasestartautomaticallywhentheArangoDBserverstarts,usetheautoStartattribute.
SettingtheadaptivePollingattributetotruewillmakethereplicationapplierpollthemasterdatabaseforchangeswithavariable
frequency.Thereplicationapplierwillthenlowerthefrequencywhenthemasterisidle,andincreaseitwhenthemastercanprovidenew
events).Otherwisethereplicationapplierwillpollthemasterdatabaseforchangeswithaconstantfrequency.
TheidleMinWaitTimeattributecontrolstheminimumwaittime(inseconds)thatthereplicationapplierwillintentionallyidlebefore
fetchingmorelogdatafromthemasterincasethemasterhasalreadysentallitslogdata.Thiswaittimecanbeusedtocontrolthe
frequencywithwhichthereplicationappliersendsHTTPlogfetchrequeststothemasterincasethereisnowriteactivityonthemaster.
TheidleMaxWaitTimeattributecontrolsthemaximumwaittime(inseconds)thatthereplicationapplierwillintentionallyidlebefore
fetchingmorelogdatafromthemasterincasethemasterhasalreadysentallitslogdataandtherehavebeenpreviouslogfetchattempts
thatresultedinnomorelogdata.Thiswaittimecanbeusedtocontrolthemaximumfrequencywithwhichthereplicationappliersends
413
AsynchronousReplication
HTTPlogfetchrequeststothemasterincasethereisnowriteactivityonthemasterforlongerperiods.Notethatthisconfiguration
valuewillonlybeusediftheoptionadaptivePollingissettotrue.
Tosetatimeoutforconnectionandfollowingrequestattempts,usetheconnectTimeoutandrequestTimeoutvalues.The
maxConnectRetriesattributeconfiguresafterhowmanyfailedconnectionattemptsinarowthereplicationapplierwillgiveupandturn
itselfoff.Youmaywanttosetthistoahighvaluesothattemporarynetworkoutagesdonotleadtothereplicationapplierstopping
itself.TheconnectRetryWaitTimeattributeconfigureshowlongthereplicationapplierwillwaitbeforeretryingtheconnectiontothe
masterincaseofconnectionproblems.
ThechunkSizeattributecanbeusedtocontroltheapproximatemaximumsizeofamaster'sresponse(inbytes).Settingittoalowvalue
maymakethemasterrespondfaster(lessdataisassembledbeforethemastersendstheresponse),butmayrequiremorerequestresponseroundtrips.Setitto0touseArangoDB'sbuilt-indefaultvalue.
TheincludeSystemattributecontrolswhetherchangestosystemcollections(suchas_graphsor_users)shouldbeapplied.Ifsettotrue,
changesinthesecollectionswillbereplicated,otherwise,theywillnotbereplicated.Itisoftennotnecessarytoreplicatedatafrom
systemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausetheslaveneedstohaveitsownsystemcollectionsin
ordertostartandkeepoperational.
TherequireFromPresentattributecontrolswhethertheapplierwillstartsynchronizingincaseitdetectsthatthemastercannotprovide
datafortheinitialtickvalueprovidedbytheslave.Thismaybethecaseifthemasterdoesnothaveabigenoughbacklogofhistoric
WALlogfiles,andwhenthereplicationisre-startedafteralongerpause.WhenrequireFromPresentissettotrue,thenthereplication
applierwillcheckatstartwhetherthestarttickfromwhichitstartsorresumesreplicationisstillpresentonthemaster.Ifnot,thenthere
wouldbedataloss.IfrequireFromPresentistrue,thereplicationapplierwillabortwithanappropriateerrormessage.Ifsettofalse,
thenthereplicationapplierwillstillstart,andignorethedataloss.
TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentand
autoResyncaresettotrueandthemastercannotprovidethelogdatatheslaverequests,thereplicationapplierwillstopasusual.Butdue
tothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,the
replicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullresynchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.
Automaticre-synchronizationmaytransferalotofdatafromthemastertotheslaveandcanbeexpensive.Itisthereforeturnedoffby
default.Whenturnedoff,theslavewillneverperformanautomaticre-synchronizationwiththemaster.
TheautoResyncRetriesoptioncanbeusedtocontrolthenumberofresynchronizationretriesthatwillbeperformedinarowwhen
automaticresynchronizationisenabledandkicksin.Settingthisto0willeffectivelydisableautoResync.Settingittosomeothervalue
willlimitthenumberofretriesthatareperformed.Thishelpspreventingendlessretriesincaseresynchronizationsalwaysfail.
Theverboseattributecontrolstheverbosityofthereplicationlogger.Settingitto truewillmakethereplicationapplierwritealineto
thelogforeveryoperationitperforms.Thisshouldonlybeusedfordiagnosingreplicationproblems.
Thefollowingexamplewillsetmostofthediscussedpropertiesforthecurrentdatabase'sapplier:
require("@arangodb/replication").applier.properties({
endpoint:"tcp://master.domain.org:8529",
username:"root",
password:"secret",
adaptivePolling:true,
connectTimeout:15,
maxConnectRetries:100,
chunkSize:262144,
autoStart:true,
includeSystem:true,
autoResync:true,
autoResyncRetries:2,
});
Aftertheapplierisnowfullyconfigured,itcouldtheoreticallybestarted.However,wemayfirstneedaninitialsynchronizationofall
collectionsandtheirdatafromthemasterbeforewestartthereplicationapplier.
Theonlysafemethodfordoingafullsynchronization(orre-synchronization)isthusto
stopthereplicationapplierontheslave(ifcurrentlyrunning)
performaninitialfullsyncwiththemasterdatabase
notethemasterdatabase'slastLogTickvalueand
414
AsynchronousReplication
startthecontinuousreplicationapplierontheslaveusingthistickvalue.
Theinitialsynchronizationforthecurrentdatabaseisexecutedwiththesynccommand:
require("@arangodb/replication").sync({
endpoint:"tcp://master.domain.org:8529",
username:"root",
password:"secret,
includeSystem:true
});
TheincludeSystemoptioncontrolswhetherdatafromsystemcollections(suchas_graphsand_users)shallbesynchronized.
TheinitialsynchronizationcanoptionallybeconfiguredtoincludeorexcludespecificcollectionsusingtherestrictTypeand
restrictCollectionparameters.
Thefollowingcommandonlysynchronizescollectionfooandbar:
require("@arangodb/replication").sync({
endpoint:"tcp://master.domain.org:8529",
username:"root",
password:"secret,
restrictType:"include",
restrictCollections:["foo","bar"]
});
UsingarestrictTypeofexclude,allcollectionsbutthespecifiedwillbesynchronized.
Warning:syncwilldoafullsynchronizationofthecollectionsinthecurrentdatabasewithcollectionspresentinthemasterdatabase.
Anylocalinstancesofthecollectionsandalltheirdataareremoved!Onlyexecutethiscommandifyouaresureyouwanttoremovethe
localdata!
Assyncdoesafullsynchronization,itmighttakeawhiletoexecute.Whensynccompletessuccessfully,itreturnsanarrayofcollections
ithassynchronizedinitscollectionsattribute.Itwillalsoreturnthemasterdatabase'slastlogtickvalueatthetimethesyncwasstarted
onthemaster.ThetickvalueiscontainedinthelastLogTickattributeofthesynccommand:
{
"lastLogTick":"231848833079705",
"collections":[...]
}
Nowyoucanstartthecontinuoussynchronizationforthecurrentdatabaseontheslavewiththecommand
require("@arangodb/replication").applier.start("231848833079705");
Note:Thetickvaluesshouldbetreatedasstrings.Usingnumericdatatypesfortickvaluesisunsafebecausetheymightexceedthe32bit
valueandtheIEEE754doubleaccuracyranges.
415
AsynchronousReplication
ExampleSetup
Settingupaworkingmaster-slavereplicationrequirestwoArangoDBinstances:
master:thisistheinstancethatalldata-modificationoperationsshouldbedirectedto
slave:onthisinstance,we'llstartareplicationapplier,andthiswillfetchdatafromthemasterdatabase'swrite-aheadlogandapply
itsoperationslocally
Forthefollowingexamplesetup,we'llusetheinstancetcp://master.domain.org:8529asthemaster,andtheinstance
tcp://slave.domain.org:8530asaslave.
Thegoalistohavealldatafromthedatabase_systemonmastertcp://master.domain.org:8529bereplicatedtothedatabase_systemon
theslavetcp://slave.domain.org:8530.
Onthemaster,nothingspecialneedstobedone,asallwriteoperationswillautomaticallybeloggedinthemaster'swrite-aheadlog
(WAL).
All-in-onesetup
Tomakethereplicationcopytheinitialdatafromthemastertotheslaveandstartthecontinuousreplicationontheslave,thereisan
all-in-onecommand:
require("@arangodb/replication").setupReplication(configuration);
Thefollowingexampledemonstrateshowtousethecommandforsettingupreplicationforthe_systemdatabase.Notethatitshouldbe
runontheslaveandnotthemaster:
db._useDatabase("_system");
require("@arangodb/replication").setupReplication({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
verbose:false,
includeSystem:false,
incremental:true,
autoResync:true
});
Thecommandwillreturnwhentheinitialsynchronizationisfinishedandthecontinuousreplicationhasbeenstarted,orincasetheinitial
synchronizationhasfailed.
Iftheinitialsynchronizationissuccessful,thecommandwillstorethegivenconfigurationontheslave.Italsoconfiguresthecontinuous
replicationtostartautomaticallyiftheslaveisrestarted,i.e.autoStartissettotrue.
Ifthecommandisrunwhiletheslave'sreplicationapplierisalreadyrunning,itwillfirststoptherunningapplier,dropitsconfiguration
anddoaresynchronizationofdatawiththemaster.Itwillthenusetheprovidedconfigration,overwritinganypreviouslyexisting
replicationconfigurationontheslave.
Initialsynchronization
Theinitialsynchronizationandcontinuousreplicationappliercanalsobestartedseparately.Tostartreplicationontheslave,makesure
therecurrentlyisnoreplicationapplierrunning.
Thefollowingcommandsstoparunningapplierintheslave's_systemdatabase:
db._useDatabase("_system");
require("@arangodb/replication").applier.stop();
416
AsynchronousReplication
Thestopoperationwillterminateanyreplicationactivityinthe_systemdatabaseontheslave.
Afterthat,theinitialsynchronizationcanberun.Itwillcopythecollectionsfromthemastertotheslave,overwritingexistingdata.To
runtheinitialsynchronization,executethefollowingcommandsontheslave:
db._useDatabase("_system");
require("@arangodb/replication").sync({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
verbose:false
});
Usernameandpasswordonlyneedtobespecifiedwhenthemasterrequiresauthentication.Tocheckwhatthesynchronizationis
currentlydoing,supplysettheverboseoptiontotrue.Ifset,thesynchronizationwillcreatelogmessageswiththecurrent
synchronizationstatus.
Warning:Thesynccommandwillreplacedataintheslavedatabasewithdatafromthemasterdatabase!Onlyexecutethesecommands
ifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!
ThesyncoperationwillreturnanattributenamedlastLogTickwhichwe'llneedtonote.Thelastlogtickwillbeusedasthestartingpoint
forsubsequentreplicationactivity.Let'sassumewegotthefollowinglastlogtick:
{
"lastLogTick":"40694126",
...
}
InitialsynchronizationfromtheArangoShell
Theinitialsynchronizationviathesynccommandmaytakealongtimetocomplete.Theshellwillblockuntiltheslavehascompletedthe
initialsynchronizationoruntilanerroroccurs.Bydefault,thesynccommandintheArangoShellwillpolltheslaveforastatusupdate
every10seconds.
Optionallythesynccommandcanbemadenon-blockingbysettingitsasyncoptiontotrue.Inthiscase,thesynccommandwillreturn
instantlywithanidstring,andtheinitialsynchronizationwillrundetachedonthemaster.Tofetchthecurrentstatusofthesyncprogress
fromtheArangoShell,thegetSyncResultfunctioncanbeusedasfollows:
db._useDatabase("_system");
varreplication=require("@arangodb/replication");
/*runcommandinasyncmode*/
varid=replication.sync({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
async:true
});
/*nowquerythestatusofouroperation*/
print(replication.getSyncResult(id));
getSyncResultwillreturnfalseaslongasthesynchronizationisnotcomplete,andreturnthesynchronizationresultotherwise.
Continuoussynchronization
Whentheinitialsynchronizationisfinished,thecontinuousreplicationappliercanbestartedusingthelastlogtickprovidedbythesync
command.Beforestartingit,thereisatleastoneconfigurationoptiontoconsider:replicationontheslavewillberunninguntiltheslave
getsshutdown.Whentheslaveservergetsrestarted,replicationwillbeturnedoffagain.Tochangethis,wefirstneedtoconfigurethe
slave'sreplicationapplierandsetitsautoStartattribute.
Here'sthecommandtoconfigurethereplicationapplierwithseveraloptions,includingtheautoStartattribute:
417
AsynchronousReplication
db._useDatabase("_system");
require("@arangodb/replication").applier.properties({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
autoStart:true,
autoResync:true,
autoResyncRetries:2,
adaptivePolling:true,
includeSystem:false,
requireFromPresent:false,
idleMinWaitTime:0.5,
idleMaxWaitTime:1.5,
verbose:false
});
Animportantconsiderationforreplicationiswhetherdatafromsystemcollections(suchas_graphsor_users)shouldbeapplied.The
includeSystemoptioncontrolsthat.Ifsettotrue,changesinsystemcollectionswillbereplicated.Otherwise,theywillnotbereplicated.
Itisoftennotnecessarytoreplicatedatafromsystemcollections,especiallybecauseitmayleadtoconfusionontheslavebecausethe
slaveneedstohaveitsownsystemcollectionsinordertostartandkeepoperational.
TherequireFromPresentattributecontrolswhethertheapplierwillstartsynchronizingincaseitdetectsthatthemastercannotprovide
datafortheinitialtickvalueprovidedbytheslave.Thismaybethecaseifthemasterdoesnothaveabigenoughbacklogofhistoric
WALlogfiles,andwhenthereplicationisre-startedafteralongerpause.WhenrequireFromPresentissettotrue,thenthereplication
applierwillcheckatstartwhetherthestarttickfromwhichitstartsorresumesreplicationisstillpresentonthemaster.Ifnot,thenthere
wouldbedataloss.IfrequireFromPresentistrue,thereplicationapplierwillabortwithanappropriateerrormessage.Ifsettofalse,
thenthereplicationapplierwillstillstart,andignorethedataloss.
TheautoResyncoptioncanbeusedinconjunctionwiththerequireFromPresentoptionasfollows:whenbothrequireFromPresentand
autoResyncaresettotrueandthemastercannotprovidethelogdatatheslavehadrequested,thereplicationapplierwillstopasusual.
ButduetothefactthatautoResyncissettotrue,theslavewillautomaticallytriggerafullresyncofalldatawiththemaster.Afterthat,
thereplicationapplierwillgointocontinuousreplicationmodeagain.Additionally,settingautoResynctotruewilltriggerafullresynchronizationofdatawhenthecontinuousreplicationisstartedanddetectsthatthereisnostarttickvalue.
Notethatautomaticre-synchronization(autoResyncoptionsettotrue)maytransferalotofdatafromthemastertotheslaveandcan
thereforebeexpensive.Stillit'sturnedonheresothere'slessneedformanualintervention.
TheautoResyncRetriesoptioncanbeusedtocontrolthenumberofresynchronizationretriesthatwillbeperformedinarowwhen
automaticresynchronizationisenabledandkicksin.Settingthisto0willeffectivelydisableautoResync.Settingittosomeothervalue
willlimitthenumberofretriesthatareperformed.Thishelpspreventingendlessretriesincaseresynchronizationsalwaysfail.
Nowit'stimetostartthereplicationapplierontheslaveusingthelastlogtickwegotbefore:
db._useDatabase("_system");
require("@arangodb/replication").applier.start("40694126");
Thiswillreplicatealloperationshappeninginthemaster'ssystemdatabaseandapplythemontheslave,too.
Afterthat,youshouldbeabletomonitorthestateandprogressofthereplicationapplierbyexecutingthestatecommandontheslave
server:
db._useDatabase("_system");
require("@arangodb/replication").applier.state();
Pleasenotethatstoppingthereplicationapplierontheslaveusingthestopcommandshouldbeavoided.Thereasonisthatcurrently
ongoingtransactions(thathavepartlybeenreplicatedtotheslave)willbeneedtoberestartedafterarestartofthereplicationapplier.
Stoppingandrestartingthereplicationapplierontheslaveshouldthusonlybeperformedifthereiscertaintythatthemasteriscurrently
fullyidleandalltransactionshavebeenreplicatedfully.
Notethatwhileaslavehasonlypartlyexecutedatransactionfromthemaster,itmightkeepawritelockonthecollectionsinvolvedin
thetransaction.
YoumayalsowanttocheckthemasterandslavestatesviatheHTTPAPIs(seeHTTPInterfaceforReplication).
418
AsynchronousReplication
419
AsynchronousReplication
SyncingCollections
Inordertosynchronizedataforasinglecollectionfromamastertoaslaveinstance,thereisthesyncCollectionfunction:
Itwillfetchalldocumentsofthespecifiedcollectionfromthemasterdatabaseandstoretheminthelocalinstance.Afterthe
synchronization,thecollectiondataontheslavewillbeidenticaltothedataonthemaster,providednofurtherdatachangeshappenon
themaster.Anydatachangesthatareperformedonthemasterafterthesynchronizationwasstartedwillnotbecapturedby
syncCollection,butneedtobereplicatedusingtheregularreplicationappliermechanism.
Forthefollowingexamplesetup,we'llusetheinstancetcp://master.domain.org:8529asthemaster,andtheinstance
tcp://slave.domain.org:8530asaslave.
Thegoalistohavealldatafromthecollectiontestindatabase_systemonmastertcp://master.domain.org:8529bereplicatedtothe
collectiontestindatabase_systemontheslavetcp://slave.domain.org:8530.
Onthemaster,thecollectiontestneedstobepresentinthe_systemdatabase,withanydatainit.
Totransferthiscollectiontotheslave,issuethefollowingcommandsthere:
db._useDatabase("_system");
require("@arangodb/replication").syncCollection("test",{
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd"
});
Warning:ThesyncCollectioncommandwillreplacethecollection'sdataintheslavedatabasewithdatafromthemasterdatabase!Only
executethesecommandsifyouhaveverifiedyouareonthecorrectserver,inthecorrectdatabase!
SettingtheoptionalincrementalattributeinthecalltosyncCollectionwillstartanincrementaltransferofdata.Thismaybeusefulincase
whentheslavealreadyhaspartsoralmostallofthedatainthecollectionandonlythedifferencesneedtobesynchronized.Notethatto
computethedifferencestheincrementaltransferwillbuildasortedlistofalldocumentkeysinthecollectiononboththeslaveandthe
master,whichmaystillbeexpensiveforhugecollectionsintermsofmemoryusageandruntime.Duringbuildingthelistofkeysthe
collectionwillberead-lockedonthemaster.
TheinitialSyncMaxWaitTimeattributeinthecalltosyncCollectioncontrolshowlongtheslavewillwaitforamaster'sresponse.Thiswait
timecanbeusedtocontrolafterwhattimethesynchronizationwillgiveupandfail.
ThesyncCollectioncommandmaytakealongtimetocompleteifthecollectionisbig.Theshellwillblockuntiltheslavehas
synchronizedtheentirecollectionfromthemasteroruntilanerroroccurs.Bydefault,thesyncCollectioncommandintheArangoShell
willpollforastatusupdateevery10seconds.
WhensyncCollectioniscalledfromtheArangoShell,theoptionalasyncattributecanbeusedtostartthesynchronizationasabackground
processontheslave.Ifasyncissettotrue,thecalltosyncCollectionwillreturnalmostinstantlywithanidstring.Usingthisidstring,
thestatusofthesyncjobontheslavecanbequeriedusingthegetSyncResultfunctionasfollows:
db._useDatabase("_system");
varreplication=require("@arangodb/replication");
/*runcommandinasyncmode*/
varid=replication.syncCollection("test",{
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
async:true
});
/*nowquerythestatusofouroperation*/
print(replication.getSyncResult(id));
getSyncResultwillreturnfalseaslongasthesynchronizationisnotcomplete,andreturnthesynchronizationresultotherwise.
420
AsynchronousReplication
421
AsynchronousReplication
ReplicationLimitations
ThereplicationinArangoDBhasafewlimitations.SomeoftheselimitationsmayberemovedinlaterversionsofArangoDB:
thereisnofeedbackfromtheslavestothemaster.Ifaslavecannotapplyaneventitgotfromthemaster,themasterwillhavea
differentstateofdata.Inthiscase,thereplicationapplierontheslavewillstopandreportanerror.Administratorscantheneither
"fix"theproblemorre-syncthedatafromthemastertotheslaveandstarttheapplieragain.
atthemomentitisassumedthatonlythereplicationapplierexecuteswriteoperationsonaslave.ArangoDBcurrentlydoesnot
preventusersfromcarryingouttheirownwriteoperationsonslaves,thoughthismightleadtoundefinedbehaviorandthe
replicationapplierstopping.
whenareplicationslaveasksamasterforlogevents,thereplicationmasterwillreturnallwriteoperationsforuser-defined
collections,butitwillexcludewriteoperationsforcertainsystemcollections.Thefollowingcollectionsareexcludedintentionally
fromreplication:_apps,_trx,_replication,_configuration,_jobs,_queues,_sessions,_foxxlogandallstatisticscollections.Write
operationsforthefollowingsystemcollectionscanbequeriedfromamaster:_aqlfunctions,_graphs,_users.
Foxxapplicationsconsistofdatabaseentriesandapplicationscriptsinthefilesystem.ThefilesystempartsofFoxxapplications
arenottrackedanywhereandthusnotreplicatedincurrentversionsofArangoDB.ToreplicateaFoxxapplication,itisrequiredto
copytheapplicationtotheremoteserverandinstallitthereusingthefoxx-managerutility.
masterserversdonotknowwhichslavesareorwillbeconnectedtothem.Allserversinareplicationsetuparecurrentlyonly
looselycoupled.Therecurrentlyisnowayforaclienttoquerywhichserversarepresentinareplication.
whennotusingourmesosintegrationfailovermustbehandledbyclientsorclientAPIs.
therecurrentlyisonereplicationapplierperArangoDBdatabase.Itisthusnotpossibletohaveaslaveapplyoperationsfrom
multiplemastersintothesametargetdatabase.
replicationissetuponaper-databaselevel.WhenusingArangoDBwithmultipledatabases,replicationmustbeconfigured
individuallyforeachdatabase.
thereplicationapplierissingle-threaded,butwriteoperationsonthemastermaybeexecutedinparalleliftheyaffectdifferent
collections.Thusthereplicationappliermightnotbeabletocatchupwithaverypowerfulandloadedmaster.
replicationisonlysupportedbetweenthetwoArangoDBserversrunningthesameArangoDBversion.Itiscurrentlynotpossible
toreplicatebetweendifferentArangoDBversions.
areplicationappliercannotapplydatafromitself.
422
SynchronousReplication
SynchronousReplication
Atitscoresynchronousreplicationwillreplicatewriteoperationstomultiplehosts.Thisfeatureisonlyavailablewhenoperating
ArangoDBinacluster.Wheneveracoordinatorexecutesasychronouslyreplicatedwriteoperationitwillonlybereportedtobe
successfulifitwascarriedoutonallreplicas.IncontrasttomultimasterreplicationsetupsknownfromothersystemsArangoDB's
synchronousoperationguaranteesaconsistentstateacrossthecluster.
423
SynchronousReplication
Implementation
Architectureinsidethecluster
SynchronousreplicationcanbeconfiguredpercollectionviathepropertyreplicationFactor.Synchronousreplicationrequiresaclusterto
operate.
WheneveryouspecifyareplicationFactorgreaterthan1whencreatingacollection,synchronousreplicationwillbeactivatedforthis
collection.Theclusterwilldeterminesuitableleadersandfollowersforeveryrequestedshard(numberOfShards)withinthecluster.
Whenrequestingdataofashardonlythecurrentleaderwillbeaskedwhereasfollowerswillonlykeeptheircopyinsync.Thisisdueto
thecurrentimplementationoftransactions.
Usingsynchronousreplicationalonewillguaranteeconsistencyandhighavailabiltyatthecostofreducedperformance:Writerequests
willhaveahigherlatency(duetoeverywrite-requesthavingtobeexecutedonthefollowers)andreadrequestswon'tscaleoutasonly
theleaderisbeingasked.
Inaclustersynchronousreplicationwillbemanagedbythecoordinatorsfortheclient.Thedatawillalwaysbestoredonprimaries.
ThefollowingexamplewillgiveyouanideaofhowsynchronousoperationhasbeenimplementedinArangoDB.
1. Connecttoacoordinatorviaarangosh
2. Createacollection
127.0.0.1:8530@_system>db._create("test",{"replicationFactor":2})
3. thecoordinatorwillfigureoutaleaderand1followerandcreate1shard(asthisisthedefault)
4. Insertdata
127.0.0.1:8530@_system>db.test.insert({"replication":"
"})
5. Thecoordinatorwillwritethedatatotheleader,whichinturnwillreplicateittothefollower.
6. Onlywhenbothweresuccessfultheresultisreportedtobesuccessful
{
"_id":"test/7987",
"_key":"7987",
"_rev":"7987"
}
Whenafollowerfails,theleaderwillgiveuponitafter3secondsandproceedwiththeoperation.Assoonasthefollower(orthe
networkconnectiontotheleader)isbackup,thetwowillresynchronizeandsynchronousreplicationisresumed.Thishappensall
transparentlytotheclient.
ThecurrentimplementationofArangoDBdoesnotallowchangingthereplicationFactorlater.Thisissubjecttochange.Inthemeantime
theonlywayistodumpandrestorethecollection.Acookbookrecipeforthiscanbefoundhere:
https://docs.arangodb.com/cookbook/Administration/M igrate2.8to3.0.html#controling-the-number-of-shards-and-the-replication-factor
Automaticfailover
Whenevertheleaderofashardisfailingandthereisaquerytryingtoaccessdataofthatshardthecoordinatorwillcontinuetryingto
contacttheleaderuntilittimeouts.Theinternalclustersupervisionrunningontheagencywillcheckclusterhealtheveryfewsecondsand
willtakeactionifthereisnoheartbeatfromaserverfor15seconds.Iftheleaderdoesn'tcomebackintimethesupervisionwill
reorganizetheclusterbypromotingforeachshardafollowerthatisinsyncwithitsleadertobethenewleader.Fromthenonthe
coordinatorswillcontactthenewleader.
Theprocessisbestoutlinedusinganexample:
1. Theleaderofashard(letsnameitDBServer001)isgoingdown.
2. Acoordinatorisaskedtoreturnadocument:
424
SynchronousReplication
127.0.0.1:8530@_system>db.test.document("100069")
3. ThecoordinatordetermineswhichserverisresponsibleforthisdocumentandfindsDBServer001
4. ThecoordinatortriestocontactDBServer001andtimeoutsbecauseitisnotreachable.
5. Afterashortwhilethesupervision(runninginparallelontheagency)willseethatheartbeatsfromDBServer001arenotcomingin
6. Thesupervisionpromotesoneofthefollowers(sayDBServer002)thatisinsynctobeleaderandmakesDBServer001afollower.
7. AsthecoordinatorcontinuestryingtofetchthedocumentitwillseethattheleaderchangedtoDBServer002
8. Thecoordinatortriestocontactthenewleader(DBServer002)andreturnstheresult:
{
"_key":"100069",
"_id":"test/100069",
"_rev":"513",
"replication":" "
}
9. AfterawhilethesupervisiondeclaresDBServer001tobecompletelydead.
10. AnewfollowerisdeterminedfromthepoolofDBservers.
11. Thenewfollowersyncsitsdatafromtheleaderandorderisrestored.
Pleasenotethattheremaystillbetimeouts.Dependingonwhenexactlytherequesthasbeendone(inregardtothesupervision)and
dependingonthetimeneededtoreconfiguretheclusterthecoordinatormightfailwithatimeouterror!
425
SynchronousReplication
Configuration
Requirements
SynchronousreplicationrequiresanoperationalArangoDBcluster.
Enablingsynchronousreplication
Synchronousreplicationcanbeenabledpercollection.Whencreatingacollectionyoumayspecifythenumberofreplicasusingthe
replicationFactorparameter.Thedefaultvalueissetto 1whicheffectivelydisablessynchronousreplication.
Example:
127.0.0.1:8530@_system>db._create("test",{"replicationFactor":3})
Intheabovecase,anywriteoperationwillrequire2replicastoreportsuccessfromnowon.
Preparinggrowth
Youmaycreateacollectionwithhigherreplicationfactorthanavailable.Whenadditionaldbserversbecomeavailabletheshardsare
automaticallyreplicatedtothenewlyavailablemachines.
M ultiplereplicasofthesameshardcannevercoexistonthesamedbserverinstance.
426
SatelliteCollections
SatelliteCollections
SatelliteCollectionsareanEnterpriseonlyfeature.WhendoingJoinsinanArangoDBclusterdatahastoexchangedbetweendifferent
servers.
Joinswillbeexecutedonacoordinator.Itwillprepareanexecutionplanandexecuteit.Whenexecutingthecoordinatorwillcontactall
shardsofthestartingpointofthejoinandaskfortheirdata.Thedatabaseserverscarryingoutthisoperationwillloadalltheirlocaldata
andthenasktheclusterfortheotherpartofthejoin.Thisagainwillbedistributedtoallinvolvedshardsofthisjoinpart.
Insumthisresultsinmuchnetworktrafficandslowresultsdependingoftheamountofdatathathastobesentthroughoutthecluster.
Satellitecollectionsarecollectionsthatareintendedtoaddressthisissue.
Theywillfacilitatethesynchronousreplicationandreplicateallitsdatatoalldatabaseserversthatarepartofthecluster.
ThisenablesthedatabaseserverstoexecutethatpartofanyJoinlocally.
Thisgreatlyimprovesperformanceforsuchjoinsatthecostsofincreasedstoragerequirementsandpoorerwriteperformanceonthis
data.
TocreateasatellitecollectionsetthereplicationFactorofthiscollectionto"satellite".
Usingarangosh:
arangosh>db._create("satellite",{"replicationFactor":"satellite"});
Afullexample
arangosh>varexplain=require("@arangodb/aql/explainer").explain
arangosh>db._create("satellite",{"replicationFactor":"satellite"})
arangosh>db._create("nonsatellite",{numberOfShards:8})
arangosh>db._create("nonsatellite2",{numberOfShards:8})
Let'sanalyseanormaljoinnotinvolvingsatellitecollections:
arangosh>explain("FORdocinnonsatelliteFORdoc2innonsatellite2RETURN1")
Querystring:
FORdocinnonsatelliteFORdoc2innonsatellite2RETURN1
Executionplan:
IdNodeTypeSiteEst.Comment
1SingletonNodeDBS1*ROOT
4CalculationNodeDBS1-LET#2=1/*jsonexpression*//*constassignment*/
2EnumerateCollectionNodeDBS0-FORdocINnonsatellite/*fullcollectionscan*/
12RemoteNodeCOOR0-REMOTE
13GatherNodeCOOR0-GATHER
6ScatterNodeCOOR0-SCATTER
7RemoteNodeDBS0-REMOTE
3EnumerateCollectionNodeDBS0-FORdoc2INnonsatellite2/*fullcollectionscan*/
8RemoteNodeCOOR0-REMOTE
9GatherNodeCOOR0-GATHER
5ReturnNodeCOOR0-RETURN#2
Indexesused:
none
Optimizationrulesapplied:
IdRuleName
1move-calculations-up
2scatter-in-cluster
3remove-unnecessary-remote-scatter
427
SatelliteCollections
Allshardsinvolvedqueryingthe nonsatellitecollectionwillfanoutviathecoordinatortotheshardsof nonsatellite.Insum8
shardswillopen8connectionstothecoordinatoraskingfortheresultsofthe nonsatellite2join.Thecoordinatorwillfanouttothe8
shardsof nonsatellite2.Sotherewillbequitesomenetworktraffic.
Let'snowhavealookatthesameusingsatellitecollections:
arangosh>db._query("FORdocinnonsatelliteFORdoc2insatelliteRETURN1")
Querystring:
FORdocinnonsatelliteFORdoc2insatelliteRETURN1
Executionplan:
IdNodeTypeSiteEst.Comment
1SingletonNodeDBS1*ROOT
4CalculationNodeDBS1-LET#2=1/*jsonexpression*//*constassignment*/
2EnumerateCollectionNodeDBS0-FORdocINnonsatellite/*fullcollectionscan*/
3EnumerateCollectionNodeDBS0-FORdoc2INsatellite/*fullcollectionscan,satellite*/
8RemoteNodeCOOR0-REMOTE
9GatherNodeCOOR0-GATHER
5ReturnNodeCOOR0-RETURN#2
Indexesused:
none
Optimizationrulesapplied:
IdRuleName
1move-calculations-up
2scatter-in-cluster
3remove-unnecessary-remote-scatter
4remove-satellite-joins
Inthisscenarioallshardsofnonsatellitewillbecontacted.Howeverasthejoinisasatellitejoinallshardscandothejoinlocallyasthe
dataisreplicatedtoallserversreducingthenetworkoverheaddramatically.
Caveats
Theclusterwillautomaticallykeepallsatellitecollectionsonallserversinsyncbyfacilitatingthesynchronousreplication.Thismeans
thatwritewillbeexecutedontheleaderonlyandthisserverwillcoordinatereplicationtothefollowers.Ifafollowerdoesn'tanswerin
time(duetonetworkproblems,temporaryshutdownetc.)itmayberemovedasafollower.ThisisbeingreportedtotheAgency.
Thefollower(oncebackinbusiness)willthenperiodicallychecktheAgencyandknowthatitisoutofsync.Itwillthenautomatically
catchup.Thismaytakeawhiledependingonhowmuchdatahastobesynced.Whendoingajoininvolvingthesatelliteyoucanspecify
howlongtheDBServerisallowedtowaitforsyncuntilthequeryisbeingaborted.
CheckAccessingCursorsfordetails.
DuringnetworkfailurethereisalsoaminimalchancethataquerywasproperlydistributedtotheDBServersbutthataprevioussatellite
writecouldnotbereplicatedtoafollowerandtheleaderdroppedthefollower.Thefollowerhoweveronlycheckseveryfewsecondsifit
isreallyinsyncsoitmightindeeddeliverstaleresults.
428
Sharding
Sharding
ArangoDBisorganizingitscollectiondatainshards.ShardingallowstousemultiplemachinestorunaclusterofArangoDBinstances
thattogetherconstituteasingledatabase.Thisenablesyoutostoremuchmoredata,sinceArangoDBdistributesthedataautomatically
tothedifferentservers.Inmanysituationsonecanalsoreapabenefitindatathroughput,againbecausetheloadcanbedistributedto
multiplemachines.
Shardsareconfiguredpercollectionsomultipleshardsofdataformthecollectionasawhole.Todetermineinwhichshardthedataisto
bestoredArangoDBperformsahashacrossthevalues.Bydefaultthishashisbeingcreatedfrom_key.
Toconfigurethenumberofshards:
127.0.0.1:8529@_system>db._create("sharded_collection",{"numberOfShards":4});
Toconfigurethehashingforanotherattribute:
127.0.0.1:8529@_system>db._create("sharded_collection",{"numberOfShards":4,"shardKeys":["country"]});
Thiswouldbeusefultokeepdataofeverycountryinoneshardwhichwouldresultinbetterperformanceforqueriesworkingonaper
countrybase.Youcanalsospecifymultiple shardKeys.Notehoweverthatifyouchangetheshardkeysfromtheirdefault ["_key"],
thenfindingadocumentinthecollectionbyitsprimarykeyinvolvesarequesttoeverysingleshard.Furthermore,inthiscaseonecanno
longerprescribetheprimarykeyvalueofanewdocumentbutmustusetheautomaticallygeneratedone.Thislatterrestrictioncomes
fromthefactthatensuringuniquenessoftheprimarykeywouldbeveryinefficientiftheusercouldspecifytheprimarykey.
Onwhichnodeinaclusteraparticularshardiskeptisundefined.Thereisnooptiontoconfigureanaffinitybasedoncertainshardkeys.
Uniqueindexes(hash,skiplist,persistent)onshardedcollectionsareonlyallowedifthefieldsusedtodeterminetheshardkeyarealso
includedinthelistofattributepathsfortheindex:
shardKeys
indexKeys
a
a
ok
a
b
notok
a
a,b
ok
a,b
a
notok
a,b
b
notok
a,b
a,b
ok
a,b
a,b,c
ok
a,b,c
a,b
notok
a,b,c
a,b,c
ok
429
Upgrading
GeneralUpgradeInformation
Recommendedmajorupgradeprocedure
ToupgradeanexistingArangoDB2.xto3.0pleaseusetheproceduredescribedhere.
Recommendedminorupgradeprocedure
ToupgradeanexistingArangoDBdatabasetoanewerversionofArangoDB(e.g.3.0to3.1,or3.1to3.2),thefollowingmethodis
recommended:
ChecktheCHANGELOGandthelistofincompatiblechangesforAPIorotherchangesinthenewversionofArangoDBandmake
sureyourapplicationscandealwiththem
Stopthe"old"arangodserviceorbinary
Copytheentire"old"datadirectorytoasafeplace(thatis,abackup)
InstallthenewversionofArangoDBandstarttheserverwiththe--database.auto-upgradeoptiononce.Thismightwritetothe
logfileofArangoDB,soyoumaywanttocheckthelogsforanyissuesbeforegoingon.
Startthe"new"arangodserviceorbinaryregularlyandcheckthelogsforanyissues.Whenyou'reconfidenteverythingwentwell,
youmaywanttocheckthedatabasedirectoryforanyfileswiththeending.old.ThesefilesarecreatedbyArangoDBduring
upgradesandcanbesafelyremovedmanuallylater.
Ifanythinggoeswrongduringorshortlyaftertheupgrade:
Stopthe"new"arangodserviceorbinary
Reverttothe"old"arangodbinaryandrestorethe"old"datadirectory
Startthe"old"versionagain
ItisnotsupportedtousedatafilescreatedormodifiedbyanewerversionofArangoDBwithanolderArangoDBversion.Forexample,it
isunsupportedandislikelytocauseproblemswhenusing3.2datafileswithanArangoDB3.0instance.
Switchingthestorageengine
Inordertouseadifferentstorageenginewithanexistingdatadirectory,itisrequiredtofirstcreatealogicalbackupofthedatausing
arangodump.
Afterthat,thearangodservershouldberestartedwiththedesiredstorageengineselected(thiscanbedonebysettingtheoption-server.storage-engine)andusinganon-existingdatadirectory.
Whentheserverisupandrunningwiththedesiredstorageengine,thedatacanbere-importedusingarangorestore.
430
Upgradingto3.2
UpgradingtoArangoDB3.2
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB3.2.Pleasebesurethatyouhavecheckedthelist
ofchangesin3.2beforeupgrading.
Switchingthestorageengine
Inordertouseadifferentstorageenginewithanexistingdatadirectory,itisrequiredtofirstcreatealogicalbackupofthedatausing
arangodump.Thatbackupshouldbecreatedbeforetheupgradeto3.2.
Afterthat,theArangoDBinstallationcanbeupgradedandstopped.Theservershouldthenberestartedwiththedesiredstorageengine
selected(thiscanbedonebysettingtheoption--server.storage-engine)andusinganon-existingdatadirectory.Thiswillstarttheserver
withtheselectedstorageenginebutwithnodata.
Whentheserverisupandrunning,thedatafromthelogicalbackupcanbere-importedusingarangorestore.
431
Upgradingto3.1
UpgradingtoArangoDB3.1
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB3.1.Pleasebesurethatyouhavecheckedthelist
ofchangesin3.1beforeupgrading.
432
Upgradingto3.0
UpgradingtoArangoDB3.0
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB3.0.Pleasebesurethatyouhavecheckedthelist
ofchangesin3.0beforeupgrading.
MigratingdatabasesandcollectionsfromArangoDB2.8to3.0
ArangoDB3.0doesnotprovideanautomaticupdatemechanismfordatabasedirectoriescreatedwiththe2.xbranchesofArangoDB.
InordertomigratedatafromArangoDB2.8(oranolder2.xversion)intoArangoDB3.0,itisnecessarytoexportthedatafrom2.8using
arangodump,andthenimportthedumpintoafreshArangoDB3.0with arangorestore.
Todothis,firstrunthe2.8versionof arangodumptoexportthedatabasedataintoadirectory. arangodumpwilldumpthe _system
databasebydefault.Inordertomakeitdumpmultipledatabases,itneedstobeinvokedoncepersourcedatabase,e.g.
#in2.8
arangodump--server.database_system--output-directorydump-system
arangodump--server.databasemydb--output-directorydump-mydb
...
Thatwillproduceadumpdirectoryforeachdatabasethat arangodumpiscalledfor.Iftheserverhasauthenticationturnedon,itmaybe
necessarytoprovidetherequiredcredentialswheninvoking arangodump,e.g.
arangodump--server.database_system--server.usernamemyuser--server.passwordmypasswd--output-directorydump-system
Thedumpsproducedby arangodumpcannowbeimportedintoArangoDB3.0usingthe3.0versionof arangodump:
#in3.0
arangorestore--server.database_system--input-directorydump-system
arangorestore--server.databasemydb--input-directorydump-mydb
...
arangorestorewillbydefaultfailifthetargetdatabasedoesnotexist.Itcanbetoldtocreateitautomaticallyusingtheoption --createdatabasetrue:
arangorestore--server.databasemydb--create-databasetrue--input-directorydump-mydb
Andagainitmayberequiredtoprovideaccesscredentialswheninvoking arangorestore:
arangorestore--server.databasemydb--create-databasetrue--server.usernamemyuser--server.passwordmypasswd--input-directo
rydump-system
Pleasenotethattheversionofdump/restoreshouldmatchtheserverversion,i.e.itisrequiredtodumptheoriginaldatawiththe2.8
versionof arangodumpandrestoreitwiththe3.0versionof arangorestore.
Afterthatthe3.0instanceofArangoDBwillcontainthedatabasesandcollectionsthatwerepresentinthe2.8instance.
Adjustingauthenticationinfo
AuthenticationinformationwasstoredperdatabaseinArangoDB2.8,meaningtherecouldbedifferentusersandaccesscredentialsper
database.In3.0,theusersarestoredinacentrallocationinthe _systemdatabase.Tousethesameusersetupasin2.8,itmaybe
requiredtocreateextrausersand/oradjusttheirpermissions.
Inordertodothat,pleaseconnecttothe3.0instancewithanArangoShell(thiswillconnecttothe _systemdatabasebydefault):
arangosh--server.usernamemyuser--server.passwordmypasswd
433
Upgradingto3.0
Usethefollowingcommandstocreateanewuserwithsomepasswordandgrantthemaccesstoaspecificdatabase
require("@arangodb/users").save(username,password,true);
require("@arangodb/users").grantDatabase(username,databaseName,"rw");
Forexample,tocreateauser myuserwithpassword mypasswdandgivethemaccesstodatabases mydb1and mydb2,thecommands
wouldlookasfollows:
require("@arangodb/users").save("myuser","mypasswd",true);
require("@arangodb/users").grantDatabase("myuser","mydb1","rw");
require("@arangodb/users").grantDatabase("myuser","mydb2","rw");
Existinguserscanalsobeupdated,removedorlistedusingthefollowingcommands:
/*updateusermyuserwithpasswordmypasswd*/
require("@arangodb/users").update("myuser","mypasswd",true);
/*removeusermyuser*/
require("@arangodb/users").remove("myuser");
/*listallusers*/
require("@arangodb/users").all();
Foxxapplications
Thedump/restoreproceduredescribedabovewillnotexportandre-importFoxxapplications.Inordertomovethesefrom2.8to3.0,
Foxxapplicationsshouldbeexportedaszipfilesviathe2.8webinterface.
Thezipfilescanthenbeuploadedinthe"Services"sectionintheArangoDB3.0webinterface.Applicationsmayneedtobeadjusted
manuallytorunin3.0.PleaseconsultthemigrationguideforFoxxapps.
AnalternativewayofmovingFoxxappsinto3.0istocopythesourcedirectoryofa2.8Foxxapplicationmanuallyintothe3.0Foxx
appsdirectoryforthetargetdatabase(whichisnormally /var/lib/arangodb3-apps/_db/<dbname>/buttheexactlocationisplatformspecific).
434
Upgradingto2.8
UpgradingtoArangoDB2.8
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.8.Pleasebesurethatyouhavecheckedthelist
ofchangesin2.8beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.8cannotbeusedwithearlierversions(e.g.ArangoDB2.7)anymore.
Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDB
installationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.8encountersadatabasecreatedwithearlierversionsof
ArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Databasedirectoryversion(20702)islowerthancurrentversion(208
00).
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':--------------------------------------------------------------------2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':--upgrade
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2015-12-04T17:11:17Z[31432]ERRORIndatabase'_system':--------------------------------------------------------------------2015-12-04T17:11:17Z[31432]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.8startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgrade
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe -upgradecommandoneachindividualdatabasesubfolder(named database-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith --upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackup
ofyourdatabasedirectorybeforeperformingtheupgrade.
Thelastlineoftheoutputshouldlooklikethis:
2015-12-04T17:12:15Z[31558]INFOdatabaseupgradepassed
Pleasecheckthefulloutputthe --upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeused
properly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.8regularly.
Upgradingaclusterplannedinthewebinterface
435
Upgradingto2.8
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellas
runningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministrateda
clusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchthecluster
inthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillsee
anotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillsee
theusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDB
instances.Pleaseletusknowifyourunintoproblems.
Thereisanalternativewayusingthe ArangoDBshell.Insteadofsteps3.and4.aboveyoucanlaunch arangosh,pointittothe
dispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption --server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace "root"witha
usernameand ""withapasswordthatisvalidforauthenticationwiththecluster.
UpgradingFoxxappsgeneratedbyArangoDB2.7andearlier
Theimplementationofthe requirefunctionusedtoimportmodulesinArangoDBandFoxxhaschangedinordertoimprove
compatibilitywithNode.jsmodules.
Givenanapp/servicewiththefollowinglayout:
manifest.json
controllers/
todos.js
models/
todo.js
repositories/
todos.js
node_modules/
models/
todo.js
Thefile controllers/todos.jswouldpreviouslycontainthefollowing requirecalls:
var_=require('underscore');
varjoi=require('joi');
varFoxx=require('org/arangodb/foxx');
varArangoError=require('org/arangodb').ArangoError;
varTodos=require('repositories/todos');//<--!
varTodo=require('models/todo');//<--!
Therequirepaths repositories/todosand models/todowerepreviouslyresolvedlocallyasrelativetotheapproot.
Startingwith2.8thesepathswouldinsteadberesolvedasrelativetothe node_modulesfolderortheglobalArangoDBmodulepaths
beforebeingresolvedlocallyasafallback.
Inthegivenexamplelayouttheappwouldbreakin2.8becausethemodulename models/todowouldalwaysresolveto
node_modules/models/todo.js(whichpreviouslywouldhavebeenignored)insteadofthelocal models/todo.js.
Inordertomakesuretheappstillworksin2.8,therequirecallsin controllers/todos.jswouldneedtobeadjustedtolooklikethis:
436
Upgradingto2.8
var_=require('underscore');
varjoi=require('joi');
varFoxx=require('org/arangodb/foxx');
varArangoError=require('org/arangodb').ArangoError;
varTodos=require('../repositories/todos');//<--!
varTodo=require('../models/todo');//<--!
Notethattheold"global"stylerequirecallsmaystillworkin2.8butmaybreakunexpectedlyifmoduleswithmatchingnamesare
installedglobally.
437
Upgradingto2.6
UpgradingtoArangoDB2.6
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.6.Pleasebesurethatyouhavecheckedthelist
ofchangesin2.6beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.6cannotbeusedwithearlierversions(e.g.ArangoDB2.5)anymore.
Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDB
installationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.6encountersadatabasecreatedwithearlierversionsof
ArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Databasedirectoryversion(20501)islowerthancurrentversion(2060
0).
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':---------------------------------------------------------------------2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':--upgrade
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':---------------------------------------------------------------------2015-02-17T09:43:11Z[8302]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.6startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgrade
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe -upgradecommandoneachindividualdatabasesubfolder(named database-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith --upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackup
ofyourdatabasedirectorybeforeperformingtheupgrade.
Thelastlineoftheoutputshouldlooklikethis:
2014-12-22T12:03:31Z[12026]INFOdatabaseupgradepassed
Pleasecheckthefulloutputthe --upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeused
properly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.6regularly.
Upgradingaclusterplannedinthewebinterface
438
Upgradingto2.6
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellas
runningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministrateda
clusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchthecluster
inthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillsee
anotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillsee
theusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDB
instances.Pleaseletusknowifyourunintoproblems.
Thereisanalternativewayusingthe ArangoDBshell.Insteadofsteps3.and4.aboveyoucanlaunch arangosh,pointittothe
dispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption --server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace "root"witha
usernameand ""withapasswordthatisvalidforauthenticationwiththecluster.
439
Upgradingto2.5
UpgradingtoArangoDB2.5
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.5.Pleasebesurethatyouhavecheckedthelist
ofchangesin2.5beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.5cannotbeusedwithearlierversions(e.g.ArangoDB2.4)anymore.
Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDB
installationbeforeperforminganupgrade.
In2.5wehavealsochangedthepathsforFoxxapplications.PleasealsomakesurethatyouhaveabackupofallFoxxappsinyour
javascript.app-pathand javascript.dev-app-path.ItissufficienttohavethesourcefilesforFoxxsomewhereelsesoyoucanreinstall
themonerror.Tocheckthateverythinghasworkedduringupgradeyoucouldusetheweb-interfaceApplicationstabor
unix>foxx-managerlist
forallyourdatabases.Thelistedappsshouldbeidenticalbeforeandaftertheupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.5encountersadatabasecreatedwithearlierversionsof
ArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Databasedirectoryversion(20401)islowerthancurrentversion(2050
0).
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':---------------------------------------------------------------------2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':--upgrade
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2015-02-17T09:43:11Z[8302]ERRORIndatabase'_system':---------------------------------------------------------------------2015-02-17T09:43:11Z[8302]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.5startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgrade
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption.Note:Wehave
changedFoxxfolderstructureandimplementedanupgradetasktomoveyourapplicationstothenewstructure.Inordertotellthis
upgradetasktoalsomoveyourdevelopmentFoxxappspleasemakesureyougivethedev-app-pathaswell.Ifyouhavenotused
developmentmodeforFoxxappsyoucandropthe --javascript.dev-app-path.Itisonlypossibletoupgradeonedev-app-path
togetherwithonedatafolder.
unix>arangoddata--upgrade--javascript.dev-app-pathdevapps
where dataisArangoDB'smaindatadirectoryand devappsisthedirectorywhereyoudevelopFoxxapps.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe -upgradecommandoneachindividualdatabasesubfolder(named database-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
440
Upgradingto2.5
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith --upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackup
ofyourdatabasedirectorybeforeperformingtheupgrade.
Thelastlineoftheoutputshouldlooklikethis:
2014-12-22T12:03:31Z[12026]INFOdatabaseupgradepassed
Pleasecheckthefulloutputthe --upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeused
properly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.5regularly.
Upgradingaclusterplannedinthewebinterface
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellas
runningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministrateda
clusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchthecluster
inthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillsee
anotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillsee
theusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDB
instances.Pleaseletusknowifyourunintoproblems.
Thereisanalternativewayusingthe ArangoDBshell.Insteadofsteps3.and4.aboveyoucanlaunch arangosh,pointittothe
dispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption --server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace "root"witha
usernameand ""withapasswordthatisvalidforauthenticationwiththecluster.
441
Upgradingto2.4
UpgradingtoArangoDB2.4
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.4.Pleasebesurethatyouhavecheckedthelist
ofchangesin2.4beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.4cannotbeusedwithearlierversions(e.g.ArangoDB2.3)anymore.
Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDB
installationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.4encountersadatabasecreatedwithearlierversionsof
ArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':Databasedirectoryversion(20302)islowerthancurrentversion(204
00).
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':--------------------------------------------------------------------2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':--upgrade
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2014-12-22T12:02:28Z[12001]ERRORIndatabase'_system':--------------------------------------------------------------------2014-12-22T12:02:28Z[12001]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.4startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgrade
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption:
unix>arangoddata--upgrade
where dataisArangoDB'smaindatadirectory.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe -upgradecommandoneachindividualdatabasesubfolder(named database-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith --upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackup
ofyourdatabasedirectorybeforeperformingtheupgrade.
Thelastlineoftheoutputshouldlooklikethis:
2014-12-22T12:03:31Z[12026]INFOdatabaseupgradepassed
442
Upgradingto2.4
Pleasecheckthefulloutputthe --upgraderun.Upgradingmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeused
properly.Ifnoerrorsarepresentortheyhavebeenresolvedmanually,youcanstartArangoDB2.4regularly.
Upgradingaclusterplannedinthewebinterface
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellas
runningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministrateda
clusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchthecluster
inthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillsee
anotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillsee
theusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDB
instances.Pleaseletusknowifyourunintoproblems.
Thereisanalternativewayusingthe ArangoDBshell.Insteadofsteps3.and4.aboveyoucanlaunch arangosh,pointittothe
dispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption --server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace "root"witha
usernameand ""withapasswordthatisvalidforauthenticationwiththecluster.
443
Upgradingto2.3
UpgradingtoArangoDB2.3
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.3.Pleasebesurethatyouhavecheckedthelist
ofchangesin2.3beforeupgrading.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.3cannotbeusedwithearlierversions(e.g.ArangoDB2.2)anymore.
Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDB
installationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.3encountersadatabasecreatedwithearlierversionsof
ArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':Databasedirectoryversion(2.2)islowerthancurrentversion(20300)
.
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':---------------------------------------------------------------------2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':--upgrade
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2014-11-03T15:48:06Z[2694]ERRORIndatabase'_system':---------------------------------------------------------------------2014-11-03T15:48:06Z[2694]FATALDatabase'_system'needsupgrade.Pleasestarttheserverwiththe--upgradeoption
TomakeArangoDB2.3startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgrade
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption:
unix>arangoddata--upgrade
where dataisArangoDB'smaindatadirectory.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe -upgradecommandoneachindividualdatabasesubfolder(named database-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith --upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackup
ofyourdatabasedirectorybeforeperformingtheupgrade.
Theoutputshouldlooklikethis:
2014-11-03T15:48:47Z[2708]INFOIndatabase'_system':Found24definedtask(s),5task(s)torun
2014-11-03T15:48:47Z[2708]INFOIndatabase'_system':stateprod/standalone/upgrade,tasksupdateUserModel,createStatistics,
upgradeClusterPlan,setupQueues,setupJobs
2014-11-03T15:48:48Z[2708]INFOIndatabase'_system':upgradesuccessfullyfinished
2014-11-03T15:48:48Z[2708]INFOdatabaseupgradepassed
444
Upgradingto2.3
Pleasechecktheoutputthe --upgraderun.Itmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifno
errorsarepresentortheyhavebeenresolved,youcanstartArangoDB2.3regularly.
Upgradingaclusterplannedinthewebinterface
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellas
runningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministrateda
clusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchthecluster
inthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillsee
anotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillsee
theusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDB
instances.Pleaseletusknowifyourunintoproblems.
Thereisanalternativewayusingthe ArangoDBshell.Insteadofsteps3.and4.aboveyoucanlaunch arangosh,pointittothe
dispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption --server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace "root"witha
usernameand ""withapasswordthatisvalidforauthenticationwiththecluster.
445
Upgradingto2.2
UpgradingtoArangoDB2.2
PleasereadthefollowingsectionsifyouupgradefromapreviousversiontoArangoDB2.2.
PleasenotefirstthatadatabasedirectoryusedwithArangoDB2.2cannotbeusedwithearlierversions(e.g.ArangoDB2.1)anymore.
Upgradingadatabasedirectorycannotbereverted.ThereforepleasemakesuretocreateafullbackupofyourexistingArangoDB
installationbeforeperforminganupgrade.
DatabaseDirectoryVersionCheckandUpgrade
ArangoDBwillperformadatabaseversioncheckatstartup.WhenArangoDB2.2encountersadatabasecreatedwithearlierversionsof
ArangoDB,itwillrefusetostart.Thisisintentional.
Theoutputwillthenlooklikethis:
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':Databasedirectoryversion(2.1)islowerthanserverversion(2.2).
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':--------------------------------------------------------------------2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':ItseemslikeyouhaveupgradedtheArangoDBbinary.
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':Ifthisiswhatyouwantedtodo,pleaserestartwiththe
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':--upgrade
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':optiontoupgradethedatainthedatabasedirectory.
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':Normallyyoucanusethecontrolscripttoupgradeyourdatabase
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':/etc/init.d/arangodbstop
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':/etc/init.d/arangodbupgrade
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':/etc/init.d/arangodbstart
2014-07-07T22:04:53Z[18675]ERRORIndatabase'_system':--------------------------------------------------------------------2014-07-07T22:04:53Z[18675]FATALDatabaseversioncheckfailedfor'_system'.Pleasestarttheserverwiththe--upgradeopti
on
TomakeArangoDB2.2startwithadatabasedirectorycreatedwithanearlierArangoDBversion,youmayneedtoinvoketheupgrade
procedureonce.ThiscanbedonebyrunningArangoDBfromthecommandlineandsupplyingthe --upgradeoption:
unix>arangoddata--upgrade
where dataisArangoDB'smaindatadirectory.
Note:herethesamedatabaseshouldbespecifiedthatisalsospecifiedwhenarangodisstartedregularly.Pleasedonotrunthe -upgradecommandoneachindividualdatabasesubfolder(named database-<somenumber>).
Forexample,ifyouregularlystartyourArangoDBserverwith
unix>arangodmydatabasefolder
thenrunning
unix>arangodmydatabasefolder--upgrade
willperformtheupgradeforthewholeArangoDBinstance,includingallofitsdatabases.
Startingwith --upgradewillrunadatabaseversioncheckandperformanynecessarymigrations.Asusual,youshouldcreateabackup
ofyourdatabasedirectorybeforeperformingtheupgrade.
Theoutputshouldlooklikethis:
2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':startingupgradefromversion2.1to2.2.0
2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':Found19definedtask(s),2task(s)torun
2014-07-07T22:11:30Z[18867]INFOIndatabase'_system':upgradesuccessfullyfinished
2014-07-07T22:11:30Z[18867]INFOdatabaseupgradepassed
446
Upgradingto2.2
Pleasechecktheoutputthe --upgraderun.Itmayproduceerrors,whichneedtobefixedbeforeArangoDBcanbeusedproperly.Ifno
errorsarepresentortheyhavebeenresolved,youcanstartArangoDB2.2regularly.
Upgradingaclusterplannedinthewebinterface
AclusterofArangoDBinstanceshastobeupgradedaswell.ThisinvolvesupgradingallArangoDBinstancesinthecluster,aswellas
runningtheversioncheckonthewholerunningclusterintheend.
Wehavetriedtomakethisprocedureaspainlessandconvenientforyou.Weassumethatyouplanned,launchedandadministrateda
clusterusingthegraphicalfrontendinyourbrowser.Theupgradeprocedureisthenasfollows:
1. Firstshutdownyourclusterusingthegraphicalfrontendasusual.
2. Thenupgradealldispatcherinstancesonallmachinesinyourclusterusingtheversioncheckasdescribedaboveandrestartthem.
3. Nowopentheclusterdashboardinyourbrowserbypointingittothesamedispatcherthatyouusedtoplanandlaunchthecluster
inthegraphicalfrontend.Inadditiontotheusualbuttons"Relaunch","Editclusterplan"and"Deleteclusterplan"youwillsee
anotherbuttonmarked"Upgradeandrelaunchcluster".
4. Hitthisbutton,yourclusterwillbeupgradedandlaunchedandallisdoneforyoubehindthescenes.Ifallgoeswell,youwillsee
theusualclusterdashboardafterafewseconds.Ifthereisanerror,youhavetoinspectthelogfilesofyourclusterArangoDB
instances.Pleaseletusknowifyourunintoproblems.
Thereisanalternativewayusingthe ArangoDBshell.Insteadofsteps3.and4.aboveyoucanlaunch arangosh,pointittothe
dispatcherthatyouhaveusedtoplanandlaunchtheclusterusingtheoption --server.endpoint,andexecute
arangosh>require("org/arangodb/cluster").Upgrade("root","");
Thisupgradestheclusterandlaunchesit,exactlyaswiththebuttonaboveinthegraphicalfrontend.Youhavetoreplace "root"witha
usernameand ""withapasswordthatisvalidforauthenticationwiththecluster.
447
Troubleshooting
Troubleshooting
448
arangod
Arangod
IftheArangoDBserverdoesnotstartorifyoucannotconnecttoitusingarangoshorotherclients,youcantrytofindtheproblem
causebyexecutingthefollowingsteps.Iftheserverstartsupwithoutproblemsyoucanskipthissection.
Checktheserverlogfile:Iftheserverhaswrittenalogfileyoushouldcheckitbecauseitmightcontainrelevanterrorcontext
information.
Checktheconfiguration:Theserverlooksforaconfigurationfilenamedarangod.confonstartup.Thecontentsofthisfilewillbe
usedasabaseconfigurationthatcanoptionallybeoverriddenwithcommand-lineconfigurationparameters.Youshouldcheckthe
configfileforthemostrelevantparameterssuchas:
server.endpoint:WhatIPaddressandporttobindto
logparameters:Ifandwheretolog
database.directory:Paththedatabasefilesarestoredin
Iftheconfigurationrevealsthatsomethingisnotconfiguredrighttheconfigfileshouldbeadjustedandtheserverberestarted.
Starttheservermanuallyandcheckitsoutput:Startingtheservermightfailevenbeforeloggingisactivatedsotheserverwillnot
producelogoutput.Thiscanhappeniftheserverisconfiguredtowritethelogstoafilethattheserverhasnopermissionson.In
thiscasetheservercannotloganerrortothespecifiedlogfilebutwillwriteastartuperroronstderrinstead.Startingtheserver
manuallywillalsoallowyoutooverridespecificconfigurationoptions,e.g.toturnon/offfileorscreenloggingetc.
ChecktheTCPport:Iftheserverstartsupbutdoesnotacceptanyincomingconnectionsthismightbeduetofirewallconfiguration
betweentheserverandanyclient(s).TheserverbydefaultwilllistenonTCPport8529.Pleasemakesurethisportisactually
accessiblebyotherclientsifyouplantouseArangoDBinanetworksetup.
Whenusinghostnamesintheconfigurationorwhenconnecting,pleasemakesurethehostnameisactuallyresolvable.Resolving
hostnamesmightinvokeDNS,whichcanbeasourceoferrorsonitsown.
ItisgenerallygoodadvicetonotuseDNSwhenspecifyingtheendpointsandconnectionaddresses.UsingIPaddressesinsteadwill
ruleoutDNSasasourceoferrors.Anotheralternativeistouseahostnamespecifiedinthelocal/etc/hostsfile,whichwillthen
bypassDNS.
Testifcurlcanconnect:Oncetheserverisstarted,youcanquicklyverifyifitrespondstorequestsatall.Thischeckallowsyouto
determinewhetherconnectionerrorsareclient-specificornot.Ifatleastoneclientcanconnect,itislikelythatconnectionproblems
ofotherclientsarenotduetoArangoDB'sconfigurationbutduetoclientorin-betweennetworkconfigurations.
Youcantestconnectivityusingasimplecommandsuchas:
curl--dump--XGEThttp://127.0.0.1:8529/_api/version&&echo
ThisshouldreturnaresponsewithanHTTP200statuscodewhentheserverisrunning.Ifitdoesitalsomeanstheserveris
generallyacceptingconnections.Alternativetoolstocheckconnectivityarelynxorab.
449
EmergencyConsole
EmergencyConsole
TheArangoDBdatabaseserverhastwomodesofoperation:Asaserver,whereitwillanswertoclientrequestsandasanemergency
console,inwhichyoucanaccessthedatabasedirectly.Thelatter-asthenamesuggests-shouldonlybeusedincaseofanemergency,
forexample,acorruptedcollection.Usingtheemergencyconsoleallowsyoutoissueallcommandsnormallyavailableinactionsand
transactions.Whenstartingtheserverinemergencyconsolemode,theservercannothandleanyclientrequests.
Youshouldneverstartmorethanoneserverusingthesamedatabasedirectory,independentofthemodeofoperation.Normally,
ArangoDBwillpreventyoufromdoingthisbyplacingalockfileinthedatabasedirectoryandnotallowingasecondArangoDBinstance
tousethesamedatabasedirectoryifalockfileisalreadypresent.
InCaseOfDisaster
Thefollowingcommandstartsanemergencyconsole.
Note:Neverstarttheemergencyconsoleforadatabasewhichalsohasaserverattachedtoit.Ingeneral,theArangoDBshelliswhatyou
want.
>./arangod--console--logerror/tmp/vocbase
ArangoDBshell[V8version5.0.71.39,DBversion3.x.x]
arango>1+2;
3
arango>vardb=require("@arangodb").db;db.geo.count();
703
TheemergencyconsoleprovidesaJavaScriptconsoledirectlyrunninginthearangodserverprocess.Thisallowstodebugandexamine
collectionsanddocumentsaswiththenormalArangoDBshell,butwithoutclient/servercommunication.
However,itisverylikelythatyouwillneverneedtheemergencyconsoleunlessyouareanArangoDBdeveloper.
450
DatafileDebugger
DatafileDebugger
InCaseOfDisaster
AranagoDBusesappend-onlyjournals.Datacorruptionshouldonlyoccurwhenthedatabaseserveriskilled.Inthiscase,thecorruption
shouldonlyoccurinthelastobject(s)thathavebeingwrittentothejournal.
Ifacorruptionoccurswithinanormaldatafile,thenthiscanonlyhappenifahardwarefaultoccurred.
Ifajournalordatafileiscorrupt,shutdownthedatabaseserverandstarttheprogram
unix>arango-dfdb
inordertochecktheconsistencyofthedatafilesandjournals.Thisbringsup
_________________
/\___||____/_(_)|___/\/__\/_\
//\/_`|__/_`||_|||/_\//\/__\////_\/
//_//(_||||(_||_|||__///_//\/\//_\\
/___,'\__,_|\__\__,_|_||_|_|\___|/___,'\_____/\____/
Availablecollections:
0:_structures
1:_users
2:_routing
3:_modules
4:_graphs
5:products
6:prices
*:all
Collectiontocheck:
Youcannowselectwhichdatabaseandcollectionyouwanttocheck.Afteryouselectedoneorallofthecollections,aconsistencycheck
willbeperformed.
Checkingcollection#1:_users
Database
path:/usr/local/var/lib/arangodb
Collection
name:_users
identifier:82343
Datafiles
#ofjournals:1
#ofcompactors:1
#ofdatafiles:0
Datafile
path:/usr/local/var/lib/arangodb/collection-82343/journal-1065383.db
type:journal
currentsize:33554432
maximalsize:33554432
totalused:256
#ofentries:3
status:OK
Ifthereisaproblemwithoneofthedatafiles,thenthedatabasedebuggerwillprintitandpromptforwhethertoattempttofixit.
WARNING:Thejournalwasnotclosedproperly,thelastentriesarecorrupted.
ThismighthappenArangoDBwaskilledandthelastentrieswerenot
fullywrittentodisk.
451
DatafileDebugger
Wipethelastentries(Y/N)?
IfyouanswerY,thecorruptedentrywillberemoved.
Ifyouseeacorruptioninadatafile(andnotajournal),thensomethingisterriblywrong.Thesefilesareimmutableandneverchangedby
ArangoDB.Acorruptioninsuchfileisanindicationofahard-diskfailure.
452
Arangobench
Arangobench
ArangobenchisArangoDB'sbenchmarkandtesttool.Itcanbeusedtoissuetestrequeststothedatabaseforperformanceandserver
functiontesting.Itsupportsparallelqueryingandbatchrequests.
Relatedblogposts:
M easuringArangoDBinsertperformance
Gainfactorof5usingbatchrequests
Startupoptions
--async:Sendasynchronousrequests.Thedefaultvalueisfalse.
--batch-size:Numberofoperationstosendperbatch.Use0todisablebatching(thisisthedefault).
--collection:Nameofcollectiontouseintest(onlyrelevantforteststhatinvokecollections).
--replication-factor:Incaseofacluster,thereplicationfactorofthecreatedcollections.
--number-of-shards:Incaseofacluster,thenumberofshardsofthecreatedcollections.
--wait-for-sync:ThevalueofwaitForSyncforcreatedcollections.
--complexity:Complexityvaluefortestcase(default:1).M eaningdependsontestcase.
--concurrency:Numberofparallelthreadsthatwillissuerequests(default:1).
--configuration:Readconfigurationfromfile.
--delay:Useastartupdelay.Thisisonlynecessarywhenruninseries.Thedefaultvalueisfalse.
--keep-alive:UseHTTPkeep-alive(default:true).
--progress:Showprogressofbenchmark,onevery20threquest.Settofalsetodisableintermediatelogging.Thedefaultvalueis
true.
--requests:Totalnumberofrequeststoperform(default:1000).
--server.endpoint:Serverendpointtoconnectto,consistingofprotocol,IPaddressandport.Defaultstotcp://localhost:8529.
--server.database:Databasenametousewhenconnecting(default:"_system").
--server.username:Usernametousewhenconnecting(default:"root").
--server.password:Passwordtousewhenconnecting.Don'tspecifythisoptiontogetapasswordprompt.
--server.authentication:Wetherornottoshowthepasswordpromptanduseauthenticationwhenconnectingtotheserver(default:
true).
--test-case:Nameoftestcasetoperform(default:"version").Possiblevalues:
version:requests/_api/version
document:createsdocuments
collection:createscollections
import-document:createsdocumentsviatheimportAPI
hash:Create/Read/Update/Readdocumentsindexedbyahashindex
skiplist:Create/Read/Update/Readdocumentsindexedbyaskiplist
edge:Create/Read/Updateedgedocuments
shapes:Create&Deletedocumentswithheterogeneousattributenames
shapes-append:Createdocumentswithheterogeneousattributenames
random-shapes:Create/Read/Deleteheterogeneousdocumentswithrandomvalues
crud:Create/Read/Update/Delete
crud-append:Create/Read/Update/Readagain
453
Arangobench
crud-write-read:Create/ReadDocuments
aqltrx:AQLTransactionswithdeepnestedAQL FOR-loops
counttrx:usesJStransactionstocountthedocumentsandinserttheresultagain
multitrx:multipletransactionscombiningreads&writesfromjs
multi-collection:multipletransactionscombiningreads&writesfromjsonmultiplecollections
aqlinsert:insertdocumentsviaAQL
aqlv8:executeAQLwithV8functionstoinsertrandomdocuments
--verbose:PrintoutrepliesiftheHTTPheaderindicatesDBerrors.(default:false).
Examples
arangobench
StartsArangobenchwiththedefaultuserandserverendpoint.
--test-caseversion--requests1000--concurrency1
Runsthe'version'testcasewith1000requests,withoutconcurrency.
--test-casedocument--requests1000--concurrency2
Runsthe'document'testcasewith2000requests,withtwoconcurrentthreads.
--test-casedocument--requests1000--concurrency2--asynctrue
Runsthe'document'testcasewith2000requests,withconcurrency2,withasyncrequests.
--test-casedocument--requests1000--concurrency2--batch-size10
Runsthe'document'testcasewith2000requests,withconcurrency2,usingbatchrequests.
454
Architecture
Architecture
AppendOnly/MVCC
Insteadofoverwritingexistingdocuments,ArangoDBwillcreateanewversionofmodifieddocuments.Thisiseventhecasewhena
documentgetsdeleted.Thetwobenefitsare:
Objectscanbestoredcoherentlyandcompactlyinthemainmemory.
Objectsarepreserved,isolatedwritingandreadingtransactionsallowaccessingtheseobjectsforparalleloperations.
Thesystemcollectsobsoleteversionsasgarbage,recognizingthemasforsaken.Garbagecollectionisasynchronousandrunsparallelto
otherprocesses.
MostlyMemory/Durability
Databasedocumentsarestoredinmemory-mappedfiles.Perdefault,thesememory-mappedfilesaresyncedregularlybutnotinstantly.
Thisisoftenagoodtradeoffbetweenstorageperformanceanddurability.Ifthislevelofdurabilityistoolowforanapplication,the
servercanalsosyncallmodificationstodiskinstantly.Thiswillgivefulldurabilitybutwillcomewithaperformancepenaltyaseach
datamodificationwilltriggerasyncI/Ooperation.
455
Write-aheadlog
Write-aheadlog
TheWrite-aheadlogispartoftheM M Filesstorageengine;Thisdoesn'tapplytoyourArangoDBifyouarerunningwiththeRocksDB
storageengine.
Startingwithversion2.2ArangoDBstoresalldata-modificationoperationinitswrite-aheadlog.Thewrite-aheadlogissequenceof
append-onlyfilescontainingallthewriteoperationsthatwereexecutedontheserver.
Itisusedtorundatarecoveryafteraservercrash,andcanalsobeusedinareplicationsetupwhenslavesneedtoreplaythesame
sequenceofoperationsasonthemaster.
Bydefault,eachwrite-aheadlogfileis32M iBinsize.Thissizeisconfigurableviatheoption--wal.logfile-size.
Whenawrite-aheadlogfileisfull,itissettoread-only,andfollowingoperationswillbewrittenintothenextwrite-aheadlogfile.By
default,ArangoDBwillreservesomesparelogfilesinthebackgroundsoswitchinglogfilesshouldbefast.Howmanyreservelogfiles
ArangoDBwilltrytokeepavailableinthebackgroundcanbecontrolledbytheconfigurationoption--wal.reserve-logfiles.
Datacontainedinfullwrite-aheadfileswilleventuallybetransferredintothejournalsordatafilesofcollections.Onlythe"surviving"
documentswillbecopiedover.Whenallremainingoperationsfromawrite-aheadlogfilehavebeencopiedoverintothejournalsor
datafilesofthecollections,thewrite-aheadlogfilecansafelyberemovedifitisnotusedforreplication.
Long-runningtransactionspreventwrite-aheadlogfilesfrombeingfullygarbage-collectedbecauseitisunclearwhetheratransactionwill
commitorabort.Long-runningtransactionscanthusblockthegarbage-collectionprogressandshouldthereforebeavoidedatallcosts.
Onasystemthatactsasareplicationmaster,itisusefultokeepafewofthealreadycollectedwrite-aheadlogfilessoreplicationslaves
stillcanfetchdatafromthemifrequired.Howmanycollectedlogfileswillbekeptbeforetheygetdeletedisconfigurableviatheoption-wal.historic-logfiles.
Forallwrite-aheadlogconfigurationoptions,pleaserefertothepageWrite-aheadlogoptions.
456
StorageEngines
StorageEngines
AttheverybottomoftheArangoDBdatabaseliesthestorageengine.Thestorageengineisresponsibleforpersistingthedocumentson
disk,holdingcopiesinmemory,providingindexesandcachestospeedupqueries.
Uptoversion3.1ArangoDBonlysupportedmemorymappedfiles(M M FILES)assolestorageengine.Beginningwith3.2ArangoDB
hassupportforpluggablestorageengines.ThesecondsupportedengineisRocksDBfromFacebook.
RocksDBisanembeddablepersistentkey-valuestore.Itisalogstructuredatabaseandisoptimizedforfaststorage.
TheM M FILESengineisoptimizedfortheuse-casewherethedatafitsintothemainmemory.Itallowsforveryfastconcurrentreads.
However,writesblockreadsandlockingisoncollectionlevel.Indexesarealwaysinmemoryandarerebuiltonstartup.Thisgivesbetter
performancebutimposesalongerstartuptime.
TheROCKSDBengineisoptimizedforlargedata-setsandallowsforasteadyinsertperformanceevenifthedata-setismuchlargerthan
themainmemory.Indexesarealwaysstoredondiskbutcachesareusedtospeedupperformance.RocksDBusesdocument-levellocks
allowingforconcurrentwrites.Writesdonotblockreads.Readsdonotblockwrites.
Theenginemustbeselectedforthewholeserver/cluster.Itisnotpossibletomixengines.Thetransactionhandlingandwrite-ahead-log
formatintheindividualenginesisverydifferentandthereforecannotbemixed.
RocksDB
Advantages
RocksDBisaveryflexibleenginethatcanbeconfiguredforvarioususecases.
ThemainadvantagesofRocksDBare
document-levellocks
supportforlargedata-sets
persistentindexes
Caveats
RocksDBallowsconcurrentwrites.However,whentouchingthesamedocumentawriteconflictisraised.Thiscannothappenwiththe
M M FILESengine,thereforeapplicationsthatswitchtoRocksDBneedtobepreparedthatsuchexceptioncanarise.Itispossibleto
exclusivelylockcollectionswhenexecutingAQL.Thiswillavoidwriteconflictsbutalsoinhibitsconcurrentwrites.
Currently,anotherrestrictionisduetothetransactionhandlinginRocksDB.Transactionsarelimitedintotalsize.Ifyouhavea
statementmodifyingalotofdocumentsitisnecessarytocommitdatainbetween.ThiswillbedoneautomaticallyforAQLbydefault.
Performance
RocksDBisabasedonlog-structuredmergetree.Agoodintroductioncanbefoundin:
http://www.benstopford.com/2015/02/14/log-structured-merge-trees/
https://blog.acolyer.org/2014/11/26/the-log-structured-merge-tree-lsm-tree/
Thebasicideaisthatdataisorganizedinlevelswereeachlevelisafactorlargerthantheprevious.Newdatawillresideinsmallerlevels
whileolddataismoveddowntothelargerlevels.Thisallowstosupporthighrateofinsertsoveranextendedperiod.Inprincipleitis
possiblethatthedifferentlevelsresideondifferentstoragemedia.ThesmalleronesonfastSSD,thelargeronesonbiggerspinningdisks.
RocksDBitselfprovidesalotofdifferentknobstofinetunethestorageengineaccordingtoyouruse-case.ArangoDBsupportsthemost
commononesusingtheoptionsbelow.
Performancereportsforthestorageenginecanbefoundhere:
https://github.com/facebook/rocksdb/wiki/performance-benchmarks
https://github.com/facebook/rocksdb/wiki/RocksDB-Tuning-Guide
457
StorageEngines
ArangoDBoptions
ArangoDBhasacacheforthepersistentindexesinRocksDB.Thetotalsizeofthiscacheiscontrolledbytheoption
--cache.size
RocksDBalsohasacachefortheblocksstoredondisk.Thesizeofthiscacheiscontrolledbytheoption
--rocksdb.block-cache-size
ArangoDBdistributestheavailablememoryequallybetweenthetwocachesbydefault.
ArangoDBchoosesasizeforthevariouslevelsinRocksDBthatissuitableforgeneralpurposeapplications.
RocksDBlogstrutureddatalevelshaveincreasingsize
MEM:-L0:-L1:---L2:-------...
NeworupdatedDocumentsarefirststoredinmemory.Ifthismemtablereachesthelimitgivenby
--rocksdb.write-buffer-size
itwillconvertedtoanSSTfileandinsertedatlevel0.
Thefollowingoptioncontrolsthesizeofeachlevelandthedepth.
--rocksdb.num-levelsN
LimitsthenumberoflevelstoN.Bydefaultitis7andthereisseldomareasontochangethis.Anewlevelisonlyopenedifthereistoo
muchdatainthepreviousone.
--rocksdb.max-bytes-for-level-baseB
L0willholdatmostBbytes.
--rocksdb.max-bytes-for-level-multiplierM
EachlevelisatmostM timesasmuchbytesasthepreviousone.ThereforethemaximumnumberofbytesforlevelLcanbecalculatedas
max-bytes-for-level-base*(max-bytes-for-level-multiplier^(L-1))
Future
RocksDBimposesalimitonthetransactionsize.Itisoptimizedtohandlesmalltransactionsveryefficiently,butiseffectivelylimiting
thetotalsizeoftransactions.
ArangoDBcurrentlyusesRocksDB'stransactionstoimplementtheArangoDBtransactionhandling.Thereforethesamerestrictions
applyforArangoDBtransactionswhenusingtheRocksDBengine.
WewillimprovethisbyintroducingdistributedtransactionsinafutureversionofArangoDB.Thiswillallowhandlinglargetransactions
asaseriesofsmallRocksDBtransactionsandhenceremovingthesizerestriction.
458
Releasenotes
ReleaseNotes
WhatsNew
WhatsNewin3.2
WhatsNewin3.1
WhatsNewin3.0
WhatsNewin2.8
WhatsNewin2.7
WhatsNewin2.6
WhatsNewin2.5
WhatsNewin2.4
WhatsNewin2.3
WhatsNewin2.2
WhatsNewin2.1
KnownIssues
KnownIssuesin3.2
Incompatiblechanges
AlsoseeUpgradingintheAdministrationchapter.
Incompatiblechangesin3.2
Incompatiblechangesin3.1
Incompatiblechangesin3.0
Incompatiblechangesin2.8
Incompatiblechangesin2.7
Incompatiblechangesin2.6
Incompatiblechangesin2.5
Incompatiblechangesin2.4
Incompatiblechangesin2.3
459
WhatsNewin3.2
FeaturesandImprovements
ThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB3.2.ArangoDB3.2alsocontainsseveral
bugfixesthatarenotlistedhere.
Storageengines
ArangoDB3.2offerstwostorageengines:
thealways-existingmemory-mappedfilesstorageengine
anewstorageenginebasedonRocksDB
Memory-mappedfilesstorageengine(MMFiles)
Theformerstorageengine(namedM M Filesenginehenceforth)persistsdatainmemory-mappedfiles.
Anydatachangesaredonefirstintheengine'swrite-aheadlog(WAL).TheWALisreplayedafteracrashsotheengineoffersdurability
andcrash-safety.DatafromtheWALiseventuallymovedtocollection-specificdatafiles.Thefilesarealwayswritteninanappend-only
fashion,sodatainfilesisneveroverwritten.Obsoletedatainfileswilleventuallybepurgedbybackgroundcompactionthreads.
M ostofthisengine'sindexesarebuiltinRAM .Whenacollectionisloaded,thisrequiresrebuildingtheindexesinRAM fromthedata
storedondisk.TheM M Filesenginehascollection-levellocking.
Thisstorageengineisagoodchoicewhendata(includingtheindexes)canfitintheserver'savailableRAM .IfthesizeofdataplustheinmemoryindexesexceedsthesizeoftheavailableRAM ,thenthisenginemaytrytoallocatemorememorythanavailable.Thiswilleither
maketheoperatingsystemswapoutpartsofthedata(andcausediskI/O)or,whennoswapspaceisconfigured,invoketheoperating
system'sout-of-memoryprocesskiller.
Thelockingstrategyallowsparallelreadsandisoftengoodenoughinread-mostlyworkloads.Writesneedexclusivelocksonthe
collections,sotheycanblockotheroperationsinthesamecollection.Thelockingstrategyalsoprovidestransactionalconsistencyand
isolation.
RocksDBstorageengine
TheRocksDBstorageengineisnewinArangoDB3.2.Itisdesignedtostoredatasetsthatarebiggerthantheserver'savailableRAM .It
persistsalldata(includingtheindexes)inaRocksDBinstance.
ThatmeansanydocumentreadorwriteoperationswillbeansweredbyRocksDBunderthehood.RocksDBwillservethedatafromits
ownin-RAM cachesorfromdisk.TheRocksDBenginehasawrite-aheadlog(WAL)andusesbackgroundthreadsforcompaction.It
supportsdatacompression.
TheRocksDBstorageenginehasdocument-levellocking.Readoperationsdonotblockandareneverblockedbyotheroperations.Write
operationsonlyblockwritesonthesamedocuments/indexvalues.Becausemultiplewriterscanoperateinparallelonthesamecollection,
thereisthepossibilityofwrite-writeconflicts.Ifsuchwriteconflictisdetected,oneofthewriteoperationsisabortedwitherror1200
("conflict").Clientapplicationscantheneitheraborttheoperationorretry,basedontherequiredconsistencysemantics.
Storageengineselection
ThestorageenginetouseinanArangoDBclusterorasingle-serverinstancemustbeselectedinitially.Thedefaultstorageenginein
ArangoDB3.2istheM M Filesengineifnostorageengineisselectedexplicitly.Thisensuresallusersupgradingfromearlierversionscan
continuewiththewell-knownM M Filesengine.
Toselectthestorage-engine,thereistheconfigurationoption --server.storage-engine.Itcanbesettoeither mmfiles, rocksdbor
auto.Whilethefirsttwovalueswillexplicitlyselectastorageengine,the autooptionwillautomaticallychoosethestorageengine
basedonwhichstorageenginewaspreviouslyselected.Ifnoenginewasselectedpreviously, autowillselecttheM M Filesengine.Ifan
enginewaspreviouslyselected,theselectionwillbewrittentoafile ENGINEintheserver'sdatabasedirectoryandwillbereadfrom
thereatanysubsequentserverstarts.
460
WhatsNewin3.2
Oncethestorageenginewasselected,theselectioncannotbechangedbyadjusting --server.storage-engine.Inordertoswitchto
anotherstorageengine,itisrequiredtore-starttheserverwithanother(empty)databasedirectory.Inordertousedatacreatedwiththe
otherstorageengine,itisrequiredtodumpthedatafirstwiththeoldengineandrestoreitusingthenewstorageengine.Thiscanbe
achievedviainvokingarangodumpandarangorestore.
UnlikeinM ySQL,thestorageengineselectioninArangoDBisforanentireclusteroranentiresingle-serverinstance.Alldatabasesand
collectionswillusethesamestorageengine.
RocksDBstorageengine:supportedindextypes
TheexistingindexesintheRocksDBengineareallpersistent.Thefollowingindexesaresupportedthere:
primary:thistypeofindexisautomaticallycreated.Itindexes _id/ _key
edge:thisindexisautomaticallycreatedforedgecollections.Itindexes _fromand _to
hash,skiplist,persistent:theseareuser-definedindexes,Despitetheirnames,theyareneitherhashnorskiplistindexes.Theseindex
typesmaptothesameRocksDB-basedsortedindeximplementation.Thesameistrueforthe"persistent"index.Thenames"hash",
"skiplist"and"persistent"areonlyusedforcompatibilitywiththeM M Filesenginewheretheseindexesexistedinpreviousandthe
currentversionofArangoDB.
geo:user-definedindexforproximitysearches
fulltext:user-definedsortedrevertedindexonwordsoccurringindocuments
SatelliteCollections
WithSatelliteCollections,youcandefinecollectionstoshardtoaclusterandcollectionstoreplicatetoeachmachine.TheArangoDB
queryoptimizerknowswhereeachshardislocatedandsendstherequeststotheDBServersinvolved,whichthenexecutesthequery,
locally.Withthisapproach,networkhopsduringjoinoperationsonshardedcollectionscanbeavoidedandresponsetimescanbeclose
tothatofasingleinstance.
SatellitecollectionsareavailableintheEnterpriseedition.
Memorymanagement
makearangodstartwithlessV8JavaScriptcontexts
Thisspeedsuptheserverstartandmakesarangoduselessmemoryatstart.WheneveraV8contextisneededbyaFoxxactionor
someotherJavaScriptoperationandthereisnousableV8context,anewcontextwillbecreateddynamicallynow.
Upto --javascript.v8-contextsV8contextswillbecreated,sothisoptionwillchangeitsmeaning.PreviouslyasmanyV8
contextsasspecifiedbythisoptionwerecreatedatserverstart,andthenumberofV8contextsdidnotchangeatruntime.Nowup
tothisnumberofV8contextswillbeinuseatthesametime,buttheactualnumberofV8contextsisdynamic.
ThegarbagecollectorthreadwillautomaticallydeleteunusedV8contextsafterawhile.Thenumberofsparecontextswillgodown
toasfewasconfiguredinthenewoption --javascript.v8-contexts-minimum.ActuallythatmanyV8contextsarealsocreatedat
serverstart.
ThefirstfewrequestsinnewV8contextsmaytakelongerthanincontextsthathavebeentherealready.Performancemaytherefore
sufferabitfortheinitialrequestssenttoArangoDBorwhenthereareonlyfewbutperformance-criticalsituationsinwhichnew
V8contextsneedtobecreated.Ifthisisaconcern,itcaneasilybefixedbysetting --javascipt.v8-contexts-minimumand -javascript.v8-contextstoarelativelyhighvalue,whichwillguaranteethatmanynumberofV8contextstobecreatedatstartup
andkeptaroundevenwhenunused.
WaitingforanunusedV8contextwillnowalsoabortandwritealogmessageincasenoV8contextcanbeacquired/createdafter60
seconds.
thenumberofpendingoperationsinarangodcannowbelimitedtoaconfigurablenumber.Ifthisnumberisexceeded,theserverwill
nowrespondwithHTTP503(serviceunavailable).Themaximumsizeofpendingoperationsiscontrolledviathestartupoption -server.maximal-queue-size.Settingitto0means"nolimit".
461
WhatsNewin3.2
thein-memorydocumentrevisionscachewasremovedentirelybecauseitdidnotprovidetheexpectedbenefits.The3.1
implementationshadoweddocumentdatainRAM ,whichincreasedtheserver'sRAM usagebutdidnotspeedupdocument
lookupstoomuch.
Thisalsoobsoletesthestartupoptions --database.revision-cache-chunk-sizeand --database.revision-cache-target-size.
TheM M Filesenginenowdoesnotuseadocumentrevisionscachebuthasin-memoryindexesandmapsdocumentstoRAM
automaticallyviammapwhendocumentsareaccessed.TheRocksDBenginehasitsownmechanismforcachingaccessed
documents.
CommunicationLayer
HTTPresponsesreturnedbyarangodwillnowincludetheextraHTTPheader x-content-type-options:nosnifftoworkarounda
cross-sitescriptingbuginM SIE
thedefaultvaluefor --ssl.protocolwaschangedfromTLSv1toTLSv1.2.Whennotexplicitlyset,arangodandallclienttoolswill
nowuseTLSv1.2.
theJSONdatainallincomingHTTPrequestsinnowvalidatedforduplicateattributenames.
IncomingJSONdatawithduplicateattributenameswillnowberejectedasinvalid.PreviousversionsofArangoDBonlyvalidated
theuniquenessofattributenamesinsideincomingJSONforsomeAPIendpoints,butnotconsistentlyforallAPIs.
InternalJavaScriptRESTactionswillnowhidetheirstacktracestotheclientunlessinHTTPresponses.Insteadtheywillalways
logtothelogfile.
JavaScript
updatedV8versionto5.7.0.0
changeundocumentedbehaviourincaseofinvalidrevisionidsin If-Matchand If-None-Matchheadersfrom400(BAD)to412
(PRECONDITIONFAILED).
changedefaultstringtruncationlengthfrom80charactersto256charactersfor print/ printShellfunctionsinArangoShelland
arangod.Thiswillemitlongerprefixesofstringvaluesbeforetruncatingthemwith ...,whichishelpfulfordebugging.This
changeismostlyusefulwhenusingtheArangoShell(arangosh).
the @arangodbmodulenowprovidesa timefunctionwhichreturnsthecurrenttimeinsecondsasafloatingpointvaluewith
microsecondprecision.
Foxx
ThereisnowanofficialHTTPAPIformanagingservices,allowingservicestobeinstalled,modified,uninstalledandreconfigured
withouttheadministrativewebinterface.
ItisnowpossibletouploadasingleJavaScriptfileinsteadofaziparchiveifyourservicerequiresnoconfiguration,additionalfiles
orsetup.Aminimalmanifestwillbegeneratedautomaticallyuponinstallationandtheuploadedfilewillbeusedastheservice's
mainentrypoint.
DistributedGraphProcessing
Weaddedsupportforexecutingdistributedgraphalgorithmsaka Pregel.
Userscanrunarbitraryalgorithmsonanentiregraph,includinginclustermode.
Weimplementedanumberofalgorithmsforvariouswell-knowngraphmeasures:
ConnectedComponents
PageRank
ShortestPaths
CentralityM easures(CentralityandBetweeness)
462
WhatsNewin3.2
CommunityDetection(viaLabelPropagation,Speakers-ListenersLabelPropagationorDM ID)
Userscancontributetheirownalgorithms
AQL
Optimizerimprovements
GeoindexesarenowimplicitlyandautomaticallyusedwhenusingappropriateSORT/FILTERstatementsinAQL,withoutthe
needtousethesomewhatlimitedspecial-purposegeoAQLfunctions NEARor WITHIN.
ComparedtousingthespecialpurposeAQLfunctionsthisapproachhastheadvantagethatitismorecomposable,andwillalso
honorany LIMITvaluesusedintheAQLquery.
Thespecialpurpose NEARAQLfunctioncannowbesubstitutedwiththefollowingAQL(providedthereisageoindexpresenton
the doc.latitudeand doc.longitudeattributes):
FORdocingeoSort
SORTDISTANCE(doc.latitude,doc.longitude,0,0)
LIMIT5
RETURNdoc
WITHINcanbesubstitutedwiththefollowingAQL:
FORdocingeoFilter
FILTERDISTANCE(doc.latitude,doc.longitude,0,0)<2000
RETURNdoc
Miscellaneousimprovements
added REGEX_REPLACEAQLfunction
REGEX_REPLACE(text,search,replacement,caseInsensitive)→string
Replacethepatternsearchwiththestringreplacementinthestringtext,usingregularexpressionmatching.
text(string):thestringtosearchin
search(string):aregularexpressionsearchpattern
replacement(string):thestringtoreplacethesearchpatternwith
returnsstring(string):thestringtextwiththesearchregexpatternreplacedwiththereplacementstringwhereverthepattern
existsintext
addednewstartupoption --query.fail-on-warningtomakeAQLqueriesabortinsteadofcontinuingwithwarnings.
Whensettotrue,thiswillmakeanAQLquerythrowanexceptionandabortincaseawarningoccurs.Thisoptionshouldbeusedin
developmenttocatcherrorsearly.Ifsettofalse,warningswillnotbepropagatedtoexceptionsandwillbereturnedwiththequery
results.Thestartupoptioncanalsobeoverridenonaperquery-level.
theslowquerylistnowcontainsthevaluesofbindvariablesusedintheslowqueries.Bindvariablesarealsoprovidedforthe
currentlyrunningqueries.Thishelpsdebuggingsloworblockingqueriesthatusedynamiccollectionnamesviabindparameters.
AQLbreakingchangeincluster:TheSHORTEST_PATHstatementusingedgecollectionnamesinsteadofagraphnamesnow
requirestoexplicitlynamethevertexcollectionnameswithintheAQLqueryinthecluster.Itcanbedonebyadding WITH<name>
atthebeginningofthequery.
Example:
FORv,eINOUTBOUNDSHORTEST_PATH@startTO@targetedges[...]
Nowhastobe:
WITHvertices
FORv,eINOUTBOUNDSHORTEST_PATH@startTO@targetedges[...]
463
WhatsNewin3.2
Thischangeisduetoavoiddeadlocksitationsinclusteredcase.Anerrorstatingtheaboveisincluded.
Clienttools
addeddataexporttool,arangoexport.
arangoexportcanbeusedtoexportcollectionstojson,jsonlorxmlandexportagraphorcollectionstoxgmml.
added"jsonl"asinputfiletypeforarangoimp
added --translateoptionforarangoimptotranslateattributenamesfromtheinputfilestoattriubtenamesexpectedby
ArangoDB
The --translateoptioncanbespecifiedmultipletimes(oncepertranslationtobeexecuted).Thefollowingexamplerenamesthe
"id"columnfromtheinputfileto"_key",andthe"from"columnto"_from",andthe"to"columnto"_to":
arangoimp--typecsv--filedata.csv--translate"id=_key"--translate"from=_from"--translate"to=_to"
--translateworksforCSVandTSVinputsonly.
added --threadsoptiontoarangoimptospecifythenumberofparallelimportthreads
changeddefaultvalueforclienttoolsoption --server.max-packet-sizefrom128M Bto256M B.thisallowstransferringbigger
resultsetsfromtheserverwithouttheclienttoolsrejectingthemasinvalid.
Authentication
addedLDAPauthentication(Enterpriseonly)
Authorization
addedreadonlymodeforusers
collectionlevelauthorizationrights
Readmoreintheoverview.
Foxx
thecookiesessiontransportnowsupportsalloptionssupportedbythecookiemethodoftheresponseobject.
it'snowpossibletoprovideyourownversionofthe graphql-syncmodulewhenusingtheGraphQLextensionsforFoxxby
passingacopyofthemoduleusingthenewgraphqloption.
customAPIendpointscannowbetaggedusingthetagmethodtogenerateacleanerSwaggerdocumentation.
MiscellaneousChanges
arangodnowvalidatesseveralOS/environmentsettingsonstartupandwarnsifthesettingsarenon-ideal.Itadditionallywillprint
outwaystoremedytheoptions.
M ostofthechecksareexecutedonLinuxsystemsonly.
added"deduplicate"attributeforarrayindexes,whichcontrolswhetherinsertingduplicateindexvaluesfromthesamedocument
intoauniquearrayindexwillleadtoanerrorornot:
//withdeduplicate=true,whichisthedefaultvalue:
db._create("test");
db.test.ensureIndex({type:"hash",fields:["tags[*]"],deduplicate:true});
db.test.insert({tags:["a","b"]});
464
WhatsNewin3.2
db.test.insert({tags:["c","d","c"]});//willwork,becausededuplicate=true
db.test.insert({tags:["a"]});//willfail
//withdeduplicate=false
db._create("test");
db.test.ensureIndex({type:"hash",fields:["tags[*]"],deduplicate:false});
db.test.insert({tags:["a","b"]});
db.test.insert({tags:["c","d","c"]});//willnotwork,becausededuplicate=false
db.test.insert({tags:["a"]});//willfail
465
KnownIssuesin3.2
KnownIssues
ThefollowingknownissuesarepresentinthisversionofArangoDBandwillbefixedinfollow-upreleases:
ClusterDeployment
ThebetadoesnotyetsupportaDC/OSdeployment.YoucanusetheArangoDBStartertostartaclusteronbaremetal.
Read-OnlyUsers
Thebetaallowstoaddread-onlyusers.However,thesearenotyetsupported.
RocksDBstorageengine
TheRocksDBstorageengineisintentionallymissingthefollowingfeaturesthatarepresentintheM M Filesengine:
thedatafiledebugger(arango-dfdb)cannotbeusedwiththisstorageengine
RocksDBhasitsowncrashrecoverysousingthedfdbwillnotmakeanysensehere.
APIsthatreturncollectionpropertiesorfigureswillreturnslightlydifferentattributesfortheRocksDBenginethanforthe
M M Filesengine.Forexample,theattributes journalSize, doCompact, indexBucketsand isVolatilearepresentinthe
M M FilesenginebutnotintheRocksDBengine.ThememoryusagefiguresreportedforcollectionsintheRocksDBengineare
estimatevalues,whereastheyareexactfortheM M Filesengine.
theRocksDBenginedoesnotsupportsomeoperationswhichonlymakesenseinthecontextoftheM M Filesengine.Theseare:
the rotatemethodoncollections
the flushmethodforWALfiles
theRocksDBstorageenginedoesnotsupportvolatilecollections
transactionsarelimitedinsize.Transactionsthatgettoobig(intermsofnumberofoperationsinvolvedorthetotalsizeofdata
modifiedbythetransaction)willbecommittedautomatically.Effectivelythismeansthatbigusertransactionsaresplitintomultiple
smallerRocksDBtransactionsthatarecommittedindividually.TheentireusertransactionwillnotnecessarilyhaveACID
propertiesinthiscase.
Thethresholdvaluesfortransactionsizescanbeconfiguredgloballyusingthestartupoptions
--rocksdb.intermediate-commit-size:ifthesizeofalloperationsinatransactionreachesthisthreshold,thetransactionis
committedautomaticallyandanewtransactionisstarted.Thevalueisspecifiedinbytes.
--rocksdb.intermediate-commit-count:ifthenumberofoperationsinatransactionreachesthisvalue,thetransactionis
committedautomaticallyandanewtransactionisstarted.
--rocksdb.max-transaction-size:thisisanupperlimitforthetotalnumberofbytesofalloperationsinatransaction.Ifthe
operationsinatransactionconsumemorethanthisthresholdvalue,thetransactionwillautomaticallyabortwitherror32
("resourcelimitexceeded").
Itisalsopossibletooverridethesethresholdspertransaction.
Thefollowingknownissueswillberesolvedinfuturereleases:
theRocksDBengineisnotyetperformance-optimizedandpotentiallynotwellconfigured
collectionsforwhichageoindexispresentwillusecollection-levelwritelocksevenwiththeRocksDBengine.Readsfromthese
collectionscanstillbedoneinparallelbutnowrites
modifyingdocumentsinacollectionwithageoindexwillcausemultipleadditionalwritestoRocksDBformaintainingtheindex
structures
thenumberofdocumentsreportedforcollections( db.<collection>.count())maybeslightlywrongduringtransactionsifthereare
paralleltransactionsongoingforthesamecollectionthatalsomodifythenumberofdocuments
466
KnownIssuesin3.2
the anyoperationtoprovidearandomdocumentfromacollectionissupportedbytheRocksDBenginebuttheoperationhas
muchhigheralgorithmiccomplexitythanintheM M Filesengine.Itisthereforediscouragedtocallitforcasesotherthanmanual
inspectionofafewdocumentsinacollection
AQLqueriesintheclusterstillissueanextralockingHTTPrequestpershardthoughthiswouldnotbenecessaryfortheRocksDB
engineinmostcases
Installer
Upgradingfrom3.1to3.2onWindowsrequirestheusertomanuallycopythedatabasedirectorytothenewlocationandrunan
upgradeonthedatabase.PleaseconsulttheDocumentationfordetailedinstructions.
SystemIntegration
OnsomeLinuxsystemssystemdandsystemvmightreportthatthearangodbserviceisingoodconditionwhenitcouldnotbe
started.Inthiscasetheuserneedstocheck /var/log/arangodb3forfurtherinformationaboutthefailedstartup.
Startup
Wehaveseenthearangodprocesshangingduringstartupinslowerenvironments.ThereasonforthisisaraceintheV8context
initialization,andithasalreadybeenfixedinthedevelbranch.
WebUI
Editionlabel(CommunityorEnterprise)nexttotheArangoDBlogolooksblurryinSafarionM acOSX.
AQLEditor:Selecting"allresults"aslimitoptionwillreturnzeroresults,whichiswrong.
MacOSX
Storageengineisnotchangeableonanexistingdatabase.Currentlyonlytheinitialselectionofthestorageengineissupported.In
ordertouseanotherstorageengine,youhavetodeleteyourArangoDBapplication(M acApplicationFolder)and
/Users/<your_user_name>/Library/ArangoDBfolder.
OpenSSL1.1
ArangoDBhasbeentestedwithOpenSSL1.0onlyandwon'tbuildagainst1.1whencompilingonyourown.Seehereforhowto
compileonsystemsthatshipOpenSSL1.1bydefault.
467
Incompatiblechangesin3.2
IncompatiblechangesinArangoDB3.2
ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB3.2,andadjustanyclientprograms
ifnecessary.
AQL
AQLbreakingchangeincluster:TheSHORTEST_PATHstatementusingedge-collectionnamesinsteadofagraphnamenow
requirestoexplicitlynamethevertex-collectionnameswithintheAQLqueryinthecluster.Itcanbedonebyadding WITH<name>
atthebeginningofthequery.
Example:
FORv,eINOUTBOUNDSHORTEST_PATH@startTO@targetedges[...]
Nowhastobe:
WITHvertices
FORv,eINOUTBOUNDSHORTEST_PATH@startTO@targetedges[...]
Thischangeisduetoavoiddead-locksitationsinclusteredcase.Anerrorstatingtheaboveisincluded.
RESTAPI
RemovedundocumentedinternalHTTPAPI:
PUT/_api/edges
ThedocumentedGET/_api/edgesandtheundocumentedPOST/_api/edgesremainsunmodified.
changeundocumentedbehaviourincaseofinvalidrevisionidsin If-Matchand If-None-MatchheadersfromreturningHTTP
statuscode400(badrequest)toreturningHTTPstatuscode412(preconditionfailed).
theRESTAPIforfetchingthelistofcurrentlyrunningAQLqueriesandtheRESTAPIforfetchingthelistofslowAQLqueries
nowreturnanextrabindVarsattributewhichcontainsthebindparametersusedbythequeries.
ThisaffectsthereturnvaluesofthefollowingAPIendpoints:
GET/_api/query/current
GET/_api/query/slow
TheRESTAPIforretrievingindexes(GET/_api/index)nowreturnsthededuplicateattributeforeachindex
TheRESTAPIforcreatingindexes(POST/_api/index)nowacceptstheoptionaldeduplicateattribute
JavaScriptAPI
changeundocumentedbehaviourincaseofinvalidrevisionidsinJavaScriptdocumentoperationsfromreturningerrorcode1239
("illegaldocumentrevision")toreturningerrorcode1200("conflict").
the collection.getIndexes()functionnowreturnsthededuplicateattributeforeachindex
the collection.ensureIndex()functionnowacceptstheoptionaldeduplicateattribute
Foxx
JWTtokensissuedbythebuilt-inJWTsessionstoragenowcorrectlyspecifythe iatand expvaluesinsecondsratherthan
millisecondsasspecifiedintheJSONWebTokenstandard.
468
Incompatiblechangesin3.2
Thismayresultinpreviouslyexpiredtokensusingmillisecondsbeingincorrectlyaccepted.Forthisreasonitisrecommendedto
replacethesigning secretorsetthenew maxExpoptiontoareasonablevaluethatissmallerthantheoldestissuedexpiration
timestamp.
Forexamplesetting maxExpto 10**12wouldinvalidateallincorrectlyissuedtokensbefore9September2001withoutimpairing
newtokensuntiltheyear33658(atwhichpointthesetokensarehopefullynolongerrelevant).
ArangoDBrunninginstandalonemodewillcommitallservicesinthe javascript.app-pathtothedatabaseonstartup.Thismay
resultinuninstalledservicesshowingupinArangoDBiftheywerenotproperlyremovedfromthefilesystem.
ArangoDBcoordinatorsinaclusternowperformaself-healingstepduringstartuptoensureinstalledservicesareconsistentaccross
allcoordinators.WerecommendbackingupyourservicesandconfigurationbeforeupgradingtoArangoDB3.2,especiallyifyou
havemadeuseofthedevelopmentmode.
Servicesinstalledbeforeupgradingto3.2(includingservicesinstalledonalphareleasesofArangoDB3.2)areNOTpickedupbythe
coordinatorself-healingwatchdog.Thiscanbesolvedbyeitherupgrading/replacingtheseservicesorbyusingthe"commit"routeof
theFoxxservicemanagementHTTPAPI,whichcommitstheexactservicesinstalledonagivencoordinatortothecluster.New
serviceswillbepickedupautomatically.
TheformatusedbyFoxxtostoreinternalservicemetadatainthedatabasehasbeensimplifiedandexistingdocumentswillbe
updatedtothenewformat.Ifyouhavemadeanychangestothedatastoredinthe _appssystemcollection,youmaywishto
exportthesechangesastheywillbeoverwritten.
ThereisnowanofficialHTTPAPIformanagingservices.IfyouwerepreviouslyusinganyoftheundocumentedAPIsorthe
routesusedbytheadministrativewebinterfacewehighlyrecommendmigratingtothenewAPI.TheoldundocumentedHTTPAPI
formananagingservicesisdeprecatedandwillberemovedinafutureversionofArangoDB.
Althoughchangestothefilesystemoutsideofdevelopmentmodewerealreadystronglydiscouraged,thisisareminderthattheyare
nolongersupported.Allfilesgeneratedbyservices(whetherbyasetupscriptorduringnormaloperationsuchasuploads)should
eitherbestoredoutsidetheservicedirectoryorbeconsideredextremelyvolatile.
Introduceddistinctionbetween arangoUserand authorizedinFoxxrequests.Clusterinternalrequestswillneverhavean
arangoUserbutareauthorized.InearlierversionsofArangoDBpartsofthestatisticswerenotaccessiblebythecoordinators
becausetheunderlyingFoxxservicecouldn'tauthorizetherequests.Itnowcorrectlychecksthenew req.authorizedproperty.
req.arangoUserstillworksasbefore.Endusersmayusethisnewpropertyaswelltoeasilycheckifarequestisauthorizedornot
regardlessofaspecificuser.
Command-lineoptionschanged
--server.maximal-queue-sizeisnowanabsolutemaximum.Ifthequeueisfull,then503isreturned.Settingitto0means"nolimit".
Thedefaultvalueforthisoptionisnow 0.
thedefaultvaluefor --ssl.protocolhasbeenchangedfrom 4(TLSv1)to 5(TLSv1.2).
thestartupoptions --database.revision-cache-chunk-sizeand --database.revision-cache-target-sizearenowobsoleteanddo
nothing
thestartupoption --database.index-threadsoptionisnowobsolete
theoption --javascript.v8-contextsisnowanabsolutemaximum.TheservermaystartlessV8contextsforJavaScriptexecution
atstartup.IfatsomepointtheserverneedsmoreV8contextsitmaystartthemdynamically,untilthenumberofV8contexts
reachesthevalueof --javascript.v8-contexts.
theminimumnumberofV8contextstocreateatstartupcanbeconfiguredviathenewstartupoption --javascript.v8-contextsminimum.
addedcommand-lineoption --javascript.allow-admin-execute
Thisoptioncanbeusedtocontrolwhetheruser-definedJavaScriptcodeisallowedtobeexecutedonserverbysendingviaHTTPto
theAPIendpoint /_admin/executewithanauthenticateduseraccount.Thedefaultvalueis false,whichdisablestheexecution
ofuser-definedcode.Thisisalsotherecommendedsettingforproduction.Intestenvironments,itmaybeconvenienttoturnthe
optiononinordertosendarbitrarysetuporteardowncommandsforexecutionontheserver.
469
Incompatiblechangesin3.2
TheintroductionofthisoptionchangesthedefaultbehaviorofArangoDB3.2:3.2nowbydefaultdisablestheexecutionof
JavaScriptcodeviathisAPI,whereasearlierversionsallowedit.Torestoretheoldbehavior,itisnecessarytosettheoptionto
true.
470
WhatsNewin3.1
FeaturesandImprovements
ThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB3.1.ArangoDB3.1alsocontainsseveral
bugfixesthatarenotlistedhere.
SmartGraphs
ArangoDB3.1addsafirstmajorenterpriseonlyfeaturecalledSmartGraphs.SmartGraphsformanadditiontothealreadyexistinggraph
featuresandallowtoscalegraphsbeyondasinglemachinewhilekeepingalmostthesamequeryperformance.TheSmartGraphfeatureis
suggestedforallgraphdatabaseusecasesthatrequireaclusterofdatabaseserversforwhateverreason.Youcaneitherhaveagraphthat
istoolargetobestoredonasinglemachineonly.Oryoucanhaveasmallgraph,butatthesametimeneedadditionaldatawithhastobe
shardedandyouwanttokeepalloftheminthesameenvirenment.Oryousimplyusetheclusterforhigh-availability.Inalltheabove
casesSmartGraphswillsignificantlyincreasetheperformanceofgraphoperations.Formoredetailedinformationreadthismanual
section.
Dataformat
Theformatoftherevisionvaluesstoredinthe _revattributeofdocumentshasbeenchangedin3.1.Upto3.0theywerestrings
containinglargishdecimalnumbers.With3.1,revisionvaluesarestillstrings,butareactuallyencodedtimestampsofthecreationdateof
therevisionofthedocument.Thetimestampsareacquiredusingahybridlogicalclock(HLC)ontheDBserverthatholdstherevision
(fortheconceptofahybridlogicalclockseethispaper).Seethismanualsectionfordetails.
ArangoDB>=3.1canArangoDB3.0databasedirectoriesandwillsimplycontinuetousetheold _revattributevalues.Newrevisions
willbewrittenwiththenewtimestamps.
ItishighlyrecommendedtobackupallyourdatabeforeloadingadatabasedirectorythatwaswrittenbyArangoDB<=3.0intoan
ArangoDB>=3.1.
CommunicationLayer
ArangoDBupto3.0usedlibevforthecommunicationlayer.ArangoDBstartingfrom3.1usesBoostASIO.
StartingwithArangoDB3.1webegintoprovidetheVelocyStreamProtocol(vst)asaadditiontotheestablishedhttpprotocol.
Afewoptionshavechangedconcerningcommunication,pleasecheckoutIncompatiblechangesin3.1.
Cluster
Foritsinternalclustercommunicationa(bundledversion)ofcurlisnowbeingused.Thisenablesasynchronousoperationthroughoutthe
clusterandshouldimprovegeneralperformanceslightly.
Authenticationisnowsupportedwithinthecluster.
Documentrevisionscache
TheArangoDBservernowprovidesanin-memorycacheforfrequentlyaccesseddocumentrevisions.Documentsthatareaccessed
duringread/writeoperationsareloadedintotherevisionscacheautomatically,andsubsequentlyservedfromthere.
Thecachehasatotaltargetsize,whichcanbecontrolledwiththestartupoption --database.revision-cache-target-size.Oncethe
cachereachesthetargetsize,olderentriesmaybeevictedfromthecachetofreememory.Notethatthetargetsizecurrentlyisahigh
watermarkthatwilltriggercachememorygarbagecollectionifexceeded.However,ifallcachechunksarestillinusewhenthehighwater
markisreached,thecachemaystillgrowandallocatemorechunksuntilcacheentriesbecomeunusedandareallowedtobegarbagecollected.
471
WhatsNewin3.1
Thecacheismaintainedonaper-collectionbasis,thatis,memoryforthecacheisallocatedonaper-collectionbasisinchunks.Thesize
forthecachememorychunkscanbecontrolledviathestartupoption --database.revision-cache-chunk-size.Thedefaultvalueis4M B
perchunk.Biggerchunksizesallowsavingmoredocumentsperchunk,whichcanleadtomoreefficientchunkallocationandlookups,
butwillalsoleadtomemorywasteifmanychunksareallocatedandnotfullyused.Thelatterwillbethecaseifthereexistmanysmall
collectionswhichallallocatetheirownchunksbutnotfullyutilizethembecauseofthelownumberofdocuments.
AQL
Functionsadded
ThefollowingAQLfunctionshavebeenaddedin3.1:
OUTERSECTION(array1,array2,...,arrayn):returnsthevaluesthatoccuronlyonceacrossallarraysspecified.
DISTANCE(lat1,lon1,lat2,lon2):returnsthedistancebetweenthetwocoordinatesspecifiedby(lat1,lon1)and(lat2,lon2).The
distanceiscalculatedusingthehaversineformula.
JSON_STRINGIFY(value):returnsaJSONstringrepresentationofthevalue.
JSON_PARSE(value):convertsaJSON-encodedstringintoaregularobject
Indexusageintraversals
3.1allowsAQLtraversalstouseotherindexesthanjusttheedgeindex.Traversalswithfiltersonedgescannowmakeuseofmore
specificindexes.Forexample,thequery
FORv,e,pIN2OUTBOUND@start@@edge
FILTERp.edges[0].foo=="bar"
RETURN[v,e,p]
mayuseahashindexon ["_from","foo"]insteadoftheedgeindexonjust ["_from"].
Optimizerimprovements
M aketheAQLqueryoptimizerinjectfilterconditionexpressionsreferredtobyvariablesduringfilterconditionaggregation.For
example,inthefollowingquery
FORdocINcollection
LETcond1=(doc.value==1)
LETcond2=(doc.value==2)
FILTERcond1||cond2
RETURN{doc,cond1,cond2}
theoptimizerwillnowinjecttheconditionsfor cond1and cond2intothefiltercondition cond1||cond2,expandingitto
(doc.value==1)||(doc.value==2)andmakingtheseconditionsavailableforindexsearching.
Notethattheoptimizerpreviouslyalreadyinjectedsomeconditionsintootherconditions,butonlyifthevariablethatdefinedthe
conditionwasnotusedelsewhere.Forexample,thefilterconditioninthequery
FORdocINcollection
LETcond=(doc.value==1)
FILTERcond
RETURN{doc}
alreadygotoptimizedbeforebecause condwasonlyusedonceinthequeryandtheoptimizerdecidedtoinjectitintotheplacewhereit
wasused.
Thisonlyworkedforvariablesthatwerereferredtoonceinthequery.Whenavariablewasusedmultipletimes,theconditionwasnot
injectedasinthefollowingquery
FORdocINcollection
472
WhatsNewin3.1
LETcond=(doc.value==1)
FILTERcond
RETURN{doc,cond}
3.1allowsusingthisconditionsothatthequerycanuseanindexon doc.value(ifsuchindexexists).
Miscellaneousimprovements
Theperformanceofthe [*]operatorwasimprovedforcasesinwhichthisoperatordidnotuseanyfilters,projectionsand/or
offset/limits.
TheAQLqueryexecutorcannowreportthetimerequiredforloadingandlockingthecollectionsusedinanAQLquery.Whenprofiling
isenabled,itwillreportthetotalloadingandlockingtimeforthequeryinthe loadingcollectionssub-attributeofthe extra.profile
valueoftheresult.TheloadingandlockingtimecanalsobeviewintheAQLqueryeditorinthewebinterface.
AuditLog
Auditlogginghasbeenadded,seeAuditing.
Clienttools
Addedoption --skip-linesforarangoimpThisallowsskippingthefirstfewlinesfromtheimportfileincasetheCSVorTSVimport
areusedandsomeinitiallinesshouldbeskippedfromtheinput.
WebAdminInterface
TheusabilityoftheAQLeditorsignificantlyimproved.InadditiontothestandardJSONoutput,theAQLEditorisnowabletorender
queryresultsasagraphprevieworatable.FurthermoretheAQLeditordisplaysqueryprofilinginformation.
AddedanewGraphViewerinordertoexchangethetechnicallyobsoleteversion.ThenewGraphViewerisbasedonCanvasbutdoes
alsoincludeafirstWebGLimplementation(limitedfunctionality-willchangeinthefuture).ThenewGraphVieweroffersasmoothway
todiscoverandvisualizeyourgraphs.
Theshardviewinclustermodenowdisplaysaprogressindicatorwhilemovingshards.
Authentication
UptoArangoDB3.0authenticationofclientrequestswasonlypossiblewithHTTPbasicauthentication.
Startingwith3.1itisnowpossibletoalsouseaJSONWebTokens(JWT)forauthenticatingincomingrequests.
FordetailschecktheHTTPauthenticationchapter.Bothauthenticationmethodsarevalidandwillbesupportedinthenearfuture.Use
whateversuitsyoubest.
Foxx
GraphQL
ItisnoweasytogetstartedwithprovidingGraphQLAPIsinFoxx,seeFoxxGraphQL.
OAuth2
FoxxnowofficiallyprovidesamoduleforimplementingOAuth2clients,seeFoxxOAuth2.
Per-routemiddleware
473
WhatsNewin3.1
It'snowpossibletospecifymiddlewarefunctionsforaroutewhendefiningaroutehandler.Thesemiddlewarefunctionsonlyapplyto
thesinglerouteandsharetheroute'sparameterdefinitions.CheckouttheFoxxRouterdocumentationformoreinformation.
474
Incompatiblechangesin3.1
IncompatiblechangesinArangoDB3.1
ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB3.1,andadjustanyclientprograms
ifnecessary.
CommunicationLayer
TheinternalcommicationlayerisnowbasedonBoostASIO.Afewoptionsregardingthreadsandcommunicationhavebeenchanged.
Therearenolongertwodifferentthreadspools( --scheduler.threadsand --server.threads).Theoption --scheduler.threadshas
beenremoved.Thenumberofthreadsisnowcontrolledbytheoption --server.threadsonly.Bydefault --server.threadsissetto
thenumberofhyper-cores.
Asaconsequenceofthechange,thefollowing(hidden)startupoptionshavebeenremoved:
--server.extra-threads
--server.aql-threads
--server.backend
--server.show-backends
--server.thread-affinity
AQL
ThebehavioroftheAQLarraycomparisonoperatorshaschangedforemptyarrays:
ALLand ANYnowalwaysreturn falsewhentheleft-handoperandisanemptyarray.Thebehaviorfornon-emptyarraysdoes
notchange:
[]ALL==1willreturn false
[1]ALL==1willreturn true
[1,2]ALL==1willreturn false
[2,2]ALL==1willreturn false
[]ANY==1willreturn false
[1]ANY==1willreturn true
[1,2]ANY==1willreturn true
[2,2]ANY==1willreturn false
NONEnowalwaysreturns truewhentheleft-handoperandisanemptyarray.Thebehaviorfornon-emptyarraysdoesnot
change:
[]NONE==1willreturn true
[1]NONE==1willreturn false
[1,2]NONE==1willreturn false
[2,2]NONE==1willreturn true
WITHinclustertraversalsisnowmandatoryinordertoavoiddeadlocks.
Dataformatchanges
Theattribute maximalSizehasbeenrenamedto journalSizeincollectionmeta-datafiles("parameter.json").Filescontainingthe
maximalSizeattributewillstillbepickedupcorrectlyfornot-yetadjustedcollections.
Theformatoftherevisionvaluesstoredinthe _revattributeofdocumentshasbeenchangedin3.1.Upto3.0theywerestrings
containinglargishdecimalnumbers.With3.1,revisionvaluesarestillstrings,butareactuallyencodedtimestampsofthecreationdateof
therevisionofthedocument.Thetimestampsareacquiredusingahybridlogicalclock(HLC)ontheDBserverthatholdstherevision
(fortheconceptofahybridlogicalclockseethispaper).Seethismanualsectionfordetails.
475
Incompatiblechangesin3.1
ArangoDB>=3.1canArangoDB3.0databasedirectoriesandwillsimplycontinuetousetheold _revattributevalues.Newrevisions
willbewrittenwiththenewtimestamps.
ItishighlyrecommendedtobackupallyourdatabeforeloadingadatabasedirectorythatwaswrittenbyArangoDB<=3.0intoan
ArangoDB>=3.1.
Tochangeallyourold _revattributesintonewstyletimestampsyouhavetouse arangodumptodumpalldataout(usingArangoDB
3.0),anduse arangorestoreintothenewArangoDB3.1,whichisthesafestwaytoupgrade.
Thechangealsoaffectsthereturnformatof _revvaluesandotherrevisionvaluesinHTTPAPIs(seebelow).
HTTPAPIchanges
APIsadded
ThefollowingHTTPRESTAPIshavebeenaddedforonlinelogleveladjustmentoftheserver:
GET /_admin/log/levelreturnsthecurrentloglevelsettings
PUT /_admin/log/levelmodifiesthecurrentloglevelsettings
APIschanged
thefollowingRESTAPIsthatreturnrevisionidsnowmakeuseofthenewrevisionidformatintroducedin3.1.Allrevisionids
returnedwillbestringsasin3.0,buthaveadifferentinternalformat.
ThefollowingAPIsareaffected:
GET/_api/collection/{collection}/checksum: revisionattribute
GET/_api/collection/{collection}/revision: revisionattribute
allotherAPIsthatreturndocuments,whichmayincludethedocuments' _revattribute
Clientapplicationsshouldnottrytointerprettheinternalsofrevisionvalues,butonlyuserevisionvaluesforcheckingwhethertwo
revisionstringsareidentical.
thereplicationRESTAPIswillnowusetheattributename journalSizeinsteadof maximalSizewhenreturninginformation
aboutcollections.
thedefaultvaluefor keepNullhasbeenchangedfrom falseto trueforthefollowingpartialupdateoperationsforverticesand
edgesin/_api/gharial:
PATCH/_api/gharial/{graph}/vertex/{collection}/{key}
PATCH/_api/gharial/{graph}/edge/{collection}/{key}
Thevaluefor keepNullcanstillbesetexplicitlyto falsebysettingtheURLparameter keepNulltoavalueof false.
theRESTAPIfordroppingcollections(DELETE/_api/collection)nowacceptsanoptionalquerystringparameter isSystem,
whichcansetto trueinordertodropsystemcollections.Iftheparameterisnotsetornotsettotrue,theRESTAPIwillrefuse
todropsystemcollections.InpreviousversionsofArangoDB,the isSystemparameterdidnotexist,andtherewasnodistinction
betweensystemandnon-systemcollectionswhendroppingcollections.
theRESTAPIforretrievingAQLqueryresults(POST/_api/cursor)willnowreturnanadditionalsub-attribute loading
collectionsthatwillcontainthetotaltimerequiredforloadingandlockingcollectionsduringtheAQLquerywhenprofilingis
enabled.Theattributecanbefoundinthe extraresultattributeinsub-attribute loadingcollections.Theattributewillonlybe
setifprofilingwasenabledforthequery.
FoxxTesting
TheQUnitinterfacetoM ochahasbeenremoved.Thisaffectsthebehaviourofthe suite, test, before, after, beforeEachand
afterEachfunctionsinFoxxtestsuites.The suiteand testfunctionsarenowprovidedbytheTDDinterface.The before,
after, beforeEachand afterEachfunctionsarenowprovidedbytheBDDinterface.
Thisshouldnotcauseanyproblemswithexistingtestsbutmayresultinfailuresintestcasesthatpreviouslypassedforthewrong
reasons.Specificallytheexecutionorderofthe before, after,etcfunctionsnowfollowstheintendedorderandisnolongerarbitrary.
476
Incompatiblechangesin3.1
FordetailsontheexpectedbehaviourofthesefunctionsseethetestingchapterintheFoxxdocumentation.
477
WhatsNewin3.0
FeaturesandImprovements
ThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB3.0.ArangoDB3.0alsocontainsseveral
bugfixesthatarenotlistedhere.
Internaldataformatchanges
ArangoDBnowusesVelocyPackforstoringdocuments,queryresultsandtemporarilycomputedvalues.Usingasingledataformat
removedtheneedforsomedataconversionsinthecorethatslowedoperationsdownpreviously.
TheVelocyPackformatisalsoquitecompact,andreducesstoragespacerequirementsfor"small"valuessuchasboolean,integers,short
strings.ThiscanspeedupseveraloperationsinsideAQLqueries.
VelocyPackdocumententriesstoredondiskarealsoself-contained,inthesensethateachstoreddocumentwillcontainallofitsdata
typeandattributenamedescriptions.Whilethismayrequireabitmorespaceforstoringthedocuments,itremovestheoverheadof
fetchingattributenamesanddocumentlayoutfromsharedstructuresasinpreviousversionsofArangoDB.Italsosimplifiesthecode
pathsforstoringandreadingdocuments.
AQLimprovements
Syntaximprovements
LIKEstring-comparisonoperator
AQLnowprovidesa LIKEoperatorandcanbeusedtocomparestringslikethis,forexampleinsidefilterconditions:
valueLIKEsearch
Thischangemakes LIKEanAQLkeyword.Using LIKEasanattributeorcollectionnameinAQLthusrequiresquotingthenamefrom
nowon.
The LIKEoperatoriscurrentlyimplementedbycallingthealreadyexistingAQLfunction LIKE,whichalsoremainsoperationalin3.0.
Usethe LIKEfunctionincaseyouwanttosearchcase-insensitive(optionalparameter),asthe LIKEoperatoralwayscomparescasesensitive.
AQLarraycomparisonoperators
AllAQLcomparisonoperatorsnowalsoexistinanarrayvariant.Inthearrayvariant,theoperatorisprecededwithoneofthekeywords
ALL,ANYorNONE.Usingoneofthesekeywordschangestheoperatorbehaviortoexecutethecomparisonoperationforall,any,or
noneofitslefthandargumentvalues.Itisthereforeexpectedthatthelefthandargumentofanarrayoperatorisanarray.
Examples:
[1,2,3]ALLIN[2,3,4]//false
[1,2,3]ALLIN[1,2,3]//true
[1,2,3]NONEIN[3]//false
[1,2,3]NONEIN[23,42]//true
[1,2,3]ANYIN[4,5,6]//false
[1,2,3]ANYIN[1,42]//true
[1,2,3]ANY==2//true
[1,2,3]ANY==4//false
[1,2,3]ANY>0//true
[1,2,3]ANY<=1//true
[1,2,3]NONE<99//false
[1,2,3]NONE>10//true
[1,2,3]ALL>2//false
[1,2,3]ALL>0//true
[1,2,3]ALL>=3//false
["foo","bar"]ALL!="moo"//true
478
WhatsNewin3.0
["foo","bar"]NONE=="bar"//false
["foo","bar"]ANY=="foo"//true
Regularexpressionstring-comparisonoperators
AQLnowsupportstheoperators=~and!~fortestingstringsagainstregularexpressions.=~testsifastringvaluematchesaregular
expression,and!~testsifastringvaluedoesnotmatcharegularexpression.
Thetwooperatorsexpecttheirleft-handoperandstobestrings,andtheirright-handoperandstobestringscontainingvalidregular
expressionsasspecifiedbelow.
Theregularexpressionsmayconsistofliteralcharactersandthefollowingcharactersandsequences:
.–thedotmatchesanysinglecharacterexceptlineterminators.Toincludelineterminators,use [\s\S]insteadtosimulate .
withDOTALLflag.
\d–matchesasingledigit,equivalentto [0-9]
\s–matchesasinglewhitespacecharacter
\S–matchesasinglenon-whitespacecharacter
\t–matchesatabcharacter
\r–matchesacarriagereturn
\n–matchesaline-feedcharacter
[xyz]–setofcharacters.matchesanyoftheenclosedcharacters(i.e.x,yorzinthiscase
[^xyz]–negatedsetofcharacters.matchesanyothercharacterthantheenclosedones(i.e.anythingbutx,yorzinthiscase)
[x-z]–rangeofcharacters.M atchesanyofthecharactersinthespecifiedrange,e.g. [0-9A-F]tomatchanycharacterin
0123456789ABCDEF
[^x-z]–negatedrangeofcharacters.M atchesanyothercharacterthantheonesspecifiedintherange
(xyz)–definesandmatchesapatterngroup
(x|y)–matcheseitherxory
^–matchesthebeginningofthestring(e.g. ^xyz)
$–matchestheendofthestring(e.g. xyz$)
Notethatthecharacters ., *, ?, [, ], (, ), {, }, ^,and $haveaspecialmeaninginregularexpressionsandmay
needtobeescapedusingabackslash( \\).Aliteralbackslashshouldalsobeescapedusinganotherbackslash,i.e. \\\\.
Charactersandsequencesmayoptionallyberepeatedusingthefollowingquantifiers:
x*–matcheszeroormoreoccurrencesofx
x+–matchesoneormoreoccurrencesofx
x?–matchesoneorzerooccurrencesofx
x{y}–matchesexactlyyoccurrencesofx
x{y,z}–matchesbetweenyandzoccurrencesofx
x{y,}–matchesatleastyoccurencesofx
Enclosingidentifiersinforwardticks
AQLidentifierscannowoptionallybeenclosedinforwardticksinadditiontousingbackwardticks.Thisallowsconvenientwritingof
AQLqueriesinJavaScripttemplatestrings(whicharedelimitedwithbackticksthemselves),e.g.
varq=`FORdocIN´collection´RETURNdoc.´name´`;
Functionsadded
ThefollowingAQLfunctionshavebeenaddedin3.0:
REGEX_TEST(value,regex):testswhetherthestringvaluematchestheregularexpressionspecifiedinregex.Returnstrueifit
matches,andfalseotherwise.
Thesyntaxforregularexpressionsisthesameasfortheregularexpressionoperators=~and!~.
479
WhatsNewin3.0
HASH(value):Calculatesahashvalueforvalue.valueisnotrequiredtobeastring,butcanhaveanydatatype.Thecalculatedhash
valuewilltakethedatatypeofvalueintoaccount,soforexamplethenumber1andthestring"1"willhavedifferenthashvalues.
Forarraysthehashvalueswillbecrearedifthearrayscontainexactlythesamevalues(includingvaluetypes)inthesameorder.For
objectsthesamehashvalueswillbecreatediftheobjectshaveexactlythesameattributenamesandvalues(includingvaluetypes).
Theorderinwhichattributesappearinsideobjectsisnotimportantforhashing.Thehashvaluereturnedbythisfunctionisa
number.ThehashalgorithmisnotguaranteedtoremainthesameinfutureversionsofArangoDB.Thehashvaluesshouldtherefore
beusedonlyfortemporarycalculations,e.g.tocompareiftwodocumentsarethesame,orforgroupingvaluesinqueries.
TYPENAME(value):Returnsthedatatypenameofvalue.Thedatatypenamecanbeeithernull,bool,number,string,arrayor
object.
LOG(value):Returnsthenaturallogarithmofvalue.ThebaseisEuler'sconstant(2.71828...).
LOG2(value):Returnsthebase2logarithmofvalue.
LOG10(value):Returnsthebase10logarithmofvalue.
EXP(value):ReturnsEuler'sconstant(2.71828...)raisedtothepowerofvalue.
EXP2(value):Returns2raisedtothepowerofvalue.
SIN(value):Returnsthesineofvalue.
COS(value):Returnsthecosineofvalue.
TAN(value):Returnsthetangentofvalue.
ASIN(value):Returnsthearcsineofvalue.
ACOS(value):Returnsthearccosineofvalue.
ATAN(value):Returnsthearctangentofvalue.
ATAN2(y,x):Returnsthearctangentofthequotientofyandx.
RADIANS(value):Returnstheangleconvertedfromdegreestoradians.
DEGREES(value):Returnstheangleconvertedfromradianstodegrees.
Optimizerimprovements
"inline-subqueries"rule
TheAQLoptimizerrule"inline-subqueries"hasbeenadded.Thisrulecanpulloutcertainsubqueriesthatareusedasanoperandtoa
FORlooponelevelhigher,eliminatingthesubquerycompletely.Thisreducescomplexityofthequery'sexecutionplanandwilllikely
enablefurtheroptimizations.Forexample,thequery
FORiIN(
FORjIN[1,2,3]
RETURNj
)
RETURNi
willbetransformedbytheruleto:
FORiIN[1,2,3]
RETURNi
Thequery
FORnameIN(
FORdocIN_users
FILTERdoc.status==1
RETURNdoc.name
)
LIMIT2
480
WhatsNewin3.0
RETURNname
willbetransformedinto
FORtmpIN_users
FILTERtmp.status==1
LIMIT2
RETURNtmp.name
Therulewillonlyfirewhenthesubqueryisusedasanoperandtoa FORloop,andifthesubquerydoesnotcontaina COLLECTwithan
INTOvariable.
"remove-unnecessary-calculations"rule
TheAQLoptimizerrule"remove-unnecessary-calculations"nowfiresinmorecasesthaninpreviousversions.Thisruleremoves
calculationsfromexecutionplans,andbyhavinglesscalculationsdone,aquerymayexecutefasterorrequireslessmemory.
Therulewillnowremovecalculationsthatareusedexactlyonceinotherexpressions(e.g. LETa=docRETURNa.value)and
calculations,orcalculationsthatarejustreferencestoothervariables(e.g. LETa=b).
"optimize-traversals"rule
TheAQLoptimizerrule"merge-traversal-filter"wasrenamedto"optimize-traversals".Therulewillremoveunusededgeandpathresult
variablesfromthetraversalincasetheyarespecifiedinthe FORsectionofthetraversal,butnotreferencedlaterinthequery.Thissaves
constructingedgesandpathsresultsthatarenotusedlater.
AQLnowusesVelocyPackinternallyforstoringintermediatevalues.Formanyvaluetypesitcannowgetawaywithoutextramemory
allocationsandlessinternalconversions.ValuescanbepassedintointernalAQLfunctionswithoutcopyingthem.Thiscanleadto
reducedqueryexecutiontimesforqueriesthatuseC++-basedAQLfunctions.
"replace-or-with-in"and"use-index-for-sort"rules
Theserulesnowfireinsomeadditionalcases,whichallowssimplifyingindexlookupconditionsandremovingSortNodesfromexecution
plans.
Clusterstatemanagement
Thecluster'sinternalstateinformationisnowalsomanagedbyArangoDBinstances.Earlierversionsreliedonthirdpartysoftwarebeing
installedforthestoringtheclusterstate.ThestateismanagedbydedicatedArangoDBinstances,whichcanbestartedinaspecialagency
mode.Theseinstancescanoperateinadistributedfashion.Theywillautomaticallyelectoneofthemtobecometheirleader,being
responsibileforstoringthestatechangessentfromserversinthecluster.Theotherinstanceswillautomaticallyfollowtheleaderandwill
transparentlystandinshoulditbecomeunavailable.Theagencyinstancesarealsoself-organizing:theywillcontinuouslyprobeeach
otherandre-electleaders.Thecommunicationbetweentheagencyinstancesusetheconsensus-basedRAFTprotocol.
TheoperationsforstoringandretrievingclusterstateinformationarenowmuchlessexpensivefromanArangoDBclusternode
perspective,whichinturnallowsforfasterclusteroperationsthatneedtofetchorupdatetheoverallclusterstate.
_fromand _toattributesofedgesareupdatableandusablein
indexes
InArangoDBpriorto3.0theattributes _fromand _toofedgesweretreatedspeciallywhenloadingorstoringedges.Thatspecial
handlingledtotheseattributesbeingnotasflexibleasregulardocumentattributes.Forexample,the _fromand _toattributevaluesof
anexistingedgecouldnotbeupdatedoncetheedgewascreated.Nowthisispossibleviathesingle-documentAPIsandviaAQL.
Additionally,the _fromand _toattributescouldnotbeindexedinuser-definedindexes,e.g.tomakeeachcombinationof _fromand
_tounique.Finally,as _fromand _toreferencedthelinkedcollectionsbycollectionidandnotbycollectionname,theirmeaning
becameunclearonceareferencedcollectionwasdropped.Thecollectionidstoredinedgesthenbecameunusable,andwhenaccessing
481
WhatsNewin3.0
suchedgethecollectionnamepartofitwasalwaystranslatedto _undefined.
InArangoDB3.0,the _fromand _tovaluesofedgesaresavedasregularstrings.Thisallowsusing _fromand _toinuser-defined
indexes.Additionally,thisallowstoupdatethe _fromand _tovaluesofexistingedges.Furthermore,collectionsreferencedby _from
and _tovaluesmaybedroppedandre-createdlater.Any _fromand _tovaluesofedgespointingtosuchdroppedcollectionare
unaffectedbythedropoperationnow.
UnifiedAPIsforCRUDoperations
TheCRUDAPIsfordocumentsandedgehavebeenunified.EdgescannowbeinsertedandmodifiedviathesameAPIsasdocuments.
_fromand _toattributevaluescanbepassedasregulardocumentattributesnow:
db.myedges.insert({_from:"myvertices/some",_to:"myvertices/other",...});
Passing _fromand _toseparatelyasitwasrequiredinearlierversionsisnotnecessaryanymorebutwillstillwork:
db.myedges.insert("myvertices/some","myvertices/other",{...});
TheCRUDoperationsnowalsosupportbatchvariantsthatworksonarraysofdocuments/edges,e.g.
db.myedges.insert([
{_from:"myvertices/some",_to:"myvertices/other",...},
{_from:"myvertices/who",_to:"myvertices/friend",...},
{_from:"myvertices/one",_to:"myvertices/two",...},
]);
ThebatchvariantsarealsoavailableinArangoDB'sHTTPAPI.Theycanbeusedtomoreefficientlycarryoutoperationswithmultiple
documentsthantheirsingle-documentequivalents,whichrequiredoneHTTPrequestperoperation.Withthebatchoperations,the
HTTPrequest/responseoverheadcanbeamortizedacrossmultipleoperations.
Persistentindexes
ArangoDB3.0providesanexperimentalpersistentindexfeature.Persistentindexesstoretheindexvaluesondiskinsteadofin-memory
only.Thismeanstheindexesdonotneedtoberebuiltin-memorywhenacollectionisloadedorreloaded,whichshouldimprove
collectionloadingtimes.
ThepersistentindexesinArangoDBarebasedontheRocksDBengine.Tocreateapersistentindexforacollection,createanindexof
type"rocksdb"asfollows:
db.mycollection.ensureIndex({type:"rocksdb",fields:["fieldname"]});
Thepersistentindexesaresorted,sotheyallowequalitylookupsandrangequeries.Notethatthefeatureisstillhighlyexperimentaland
hassomeknowndeficiencies.Itwillbefinalizeduntilthereleaseofthe3.0stableversion.
UpgradedV8version
TheV8enginethatisusedinsideArangoDBtoexecuteJavaScriptcodehasbeenupgradedfromversion4.3.61to5.0.71.39.Thenew
versionmakesseveralmoreES6featuresavailablebydefault,including
arrowfunctions
computedpropertynames
restparameters
arraydestructuring
numericandobjectliterals
482
WhatsNewin3.0
WebAdminInterface
TheArangoDB3.0webinterfaceissignificantlyimproved.Itnowcomeswithamoreresponsivedesign,makingiteasiertouseon
differentdevices.Navigationandmenushavebeensimplified,andrelateditemshavebeenregroupedtostayclosertogetherandallow
tighterworkflows.
TheAQLqueryeditorisnowmucheasiertouse.M ultiplequeriescanbestartedandtrackedinparallel,whileresultsofearlierqueries
arestillpreserved.Queriesstillrunningcanbecanceleddirectlyfromtheeditor.TheAQLqueryeditornowallowstheusageofbind
parameterstoo,andprovidesahelperforfindingcollectionnames,AQLfunctionnamesandkeywordsquickly.
Thewebinterfacenowkeepstrackofwhethertheserverisofflineandofwhichserver-sideoperationshavebeenstartedandarestill
running.Itnowremainsusablewhilesuchlonger-runningoperationsareongoing.Italsokeepsmorestateaboutuser'schoices(e.g.
windowssizes,whetherthetreeorthecodeviewwaslastusedinthedocumenteditor).
Clusterstatisticsarenowintegratedintothewebinterfaceaswell.Additionally,amenuitem"Helpus"hasbeenaddedtoeasilyprovide
theArangoDBteamfeedbackabouttheproduct.
Thefrontendmaynowbemountedbehindareverseproxyonadifferentpath.ForthistoworktheproxyshouldsendaX-Script-Name
headercontainingthepath.
Abackendconfigurationforhaproxymightlooklikethis:
reqaddX-Script-Name:\/arangodb
Thefrontendwillrecognizethesubpathandproduceappropriatelinks.ArangoDBwillonlyacceptpathsfromtrustedfrontendproxies.
Trustedproxiesmaybeaddedonstartup:
--frontend.proxy-request-checktrue--frontend.trusted-proxy192.168.1.117
--frontend.trusted-proxymaybeanyaddressornetmask.
Todisablethecheckandblindlyacceptanyx-script-nameset--frontend.proxy-request-checktofalse.
Foxximprovements
TheFoxxframeworkhasbeencompletelyrewrittenfor3.0withanew,simplerandmorefamiliarAPI.Themostnotablechangesare:
Legacymodefor2.8services
Stuckwitholdcode?Youcancontinueusingyour2.8-compatibleFoxxserviceswith3.0byadding "engines":{"arangodb":
"^2.8.0"}(orsimilarversionrangesthatexclude3.0andup)totheservicemanifest.
Nomoreglobalvariablesandmagicalcomments
The applicationContextisnow module.context.Insteadofmagicalcommentsjustusethe summaryand descriptionmethods
todocumentyourroutes.
RepositoryandM odelhavebeenremoved
InsteadofrepositoriesjustuseArangoDBcollectionsdirectly.Forvalidationsimplyusethejoischemas(butwrappedin
joi.object())thatpreviouslylivedinsidethemodel.CollectionsandqueriesreturnplainJavaScriptobjects.
Controllershavebeenreplacedwithnestablerouters
Createrouterswith require('@arangodb/foxx/router')(),attachthemtoyourservicewith module.context.use(router).Because
routersarenolongermountedautomagically,youcanexportandimportthemlikeanyotherobject.Use router.use('/path',
subRouter)tonestroutersasdeeplyasyouwant.
Routescanbenamedandreversed
NomorememorizingURLs:addanametoyourroutelike router.get('/hello/:name',function(){...},'hello')andredirectto
thefullURLwith res.redirect(req.resolve('hello',{name:'world'})).
483
WhatsNewin3.0
Simplerexpress-likemiddleware
Ifyoualreadyknowexpress,thisshouldbefamiliar.Here'sarequestloggerinthreelinesofcode:
router.use(function(req,res,next){
varstart=Date.now();
try{next();}
finally{console.log(`${req.method}${req.url}${res.statusCode}${Date.now()-start}ms`);}
});
Sessionsandauthwithoutdependencies
Tomakeiteasiertogetstarted,thefunctionalitypreviouslyprovidedbythe simple-auth, oauth2, sessions-localand
sessions-jwtserviceshavebeenmovedintoFoxxasthe @arangodb/foxx/auth, @arangodb/foxx/oauth2and
@arangodb/foxx/sessionsmodules.
Logging
ArangoDB'sloggingisnowgroupedintotopics.Thelogverbosityandoutputfilescanbeadjustedperlogtopic.Forexample
--log.levelstartup=trace--log.levelqueries=trace--log.levelinfo
willlogmessagesconcerningstartupattracelevel,AQLqueriesattracelevelandeverythingelseatinfolevel. --log.levelcanbe
specifiedmultipletimesatstartup,forasmanytopicsasneeded.
Somerelevantlogtopicsavailablein3.0are:
collector:informationabouttheWALcollector'sstate
compactor:informationaboutthecollectiondatafilecompactor
datafiles:datafile-relatedoperations
mmap:informationaboutmemory-mappingoperations(includingmsync)
queries:executedAQLqueries,slowqueries
replication:replication-relatedinfo
requests:HTTPrequests
startup:informationaboutserverstartupandshutdown
threads:informationaboutthreads
Thisalsoallowsdirectinglogoutputtodifferentfilesbasedontopics.Forexample,tologallAQLqueriestoafile"queries.log"onecan
usetheoptions:
--log.levelqueries=trace--log.outputqueries=file:///path/to/queries.log
ToadditionallylogHTTPrequesttoafilenamed"requests.log"addtheoptions:
--log.levelrequests=info--log.outputrequests=file:///path/to/requests.log
Buildsystem
ArangoDBnowusesthecross-platformbuildsystemCM akeforallitsbuilds.Previousversionsusedtwodifferentbuildsystems,
makingdevelopmentandcontributionsharderthannecessary.Nowthebuildsystemisunified,andalltargets(Linux,Windows,M acOS)
arebuiltfromthesamesetofbuildinstructions.
Documentation
Thedocumentationhasbeenenhancedandre-organizedtobemoreintuitive.
484
WhatsNewin3.0
AnewintroductionforbeginnersshouldbringyouuptospeedwithArangoDBinlessthananhour.Additionaltopicshavebeen
introducedandwillbeextendedwithupcomingreleases.
ThetopicsAQLandHTTPAPIarenowseparatedfromthemanualforbettersearchabilityandlessconfusion.Aversionswitcher
makesiteasiertojumptotheversionofthedocsyouareinterestedin.
485
Incompatiblechangesin3.0
IncompatiblechangesinArangoDB3.0
ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB3.0,andadjustanyclientprograms
ifnecessary.
Buildsystem
BuildingArangoDB3.0fromsourcenowrequiresCM ake.
Thepre-3.0buildsystemusedaconfigure-basedapproach.ThestepstobuildArangoDB2.8fromsourcecodewere:
makesetup
./configure<options>
make
Thesestepswillnotworkanymore,asArangoDB3.0doesnotcomewithaconfigurescript.
Tobuild3.0onLinux,createaseparatebuilddirectoryfirst:
mkdir-pbuild
andthencreatetheinitialbuildscriptsonceusingCM ake:
(cdbuild&&cmake<options>..)
Theabovecommandwillconfigurethebuildandcheckfortherequireddependencies.Ifeverythingworkswelltheactualbuildcanbe
startedwith
(cdbuild&&make)
ThebinariesfortheArangoDBserverandallclienttoolswillthenbecreatedinsidethe builddirectory.TostartArangoDBlocally
fromthe builddirectory,use
build/bin/arangod<options>
Datafilesanddatafilenames
ArangoDB3.0usesanewVelocyPack-basedformatforstoringdatainWALlogfilesandcollectiondatafiles.Thefileformatisnot
compatiblewiththefilesusedinpriorversionsofArangoDB.ThatmeansdatafileswrittenbyArangoDB3.0cannotbeusedinearlier
versionsandviceversa.
Thepatternforcollectiondirectorynameswaschangedin3.0toincludearandomidcomponentattheend.Thenewpatternis
collection-<id>-<random>,where <id>isthecollectionidand <random>isarandomnumber.PreviousversionsofArangoDBuseda
pattern collection-<id>withouttherandomnumber.
UserManagement
UnlikeArangoDB2.x,ArangoDB3.0usersarenowseparatedfromdatabases,andyoucangrantoneormoredatabasepermissionstoa
user.
IfyouwanttomimicthebehaviorofArangoDB,youshouldnameyouruserslike username@dbname.
Usersthatcanaccessthe_systemdatabaseareallowedtomanageusersandpermissionsforalldatabases.
486
Incompatiblechangesin3.0
Edgesandedgesattributes
InArangoDBpriorto3.0theattributes _fromand _toofedgesweretreatedspeciallywhenloadingorstoringedges.Thatspecial
handlingledtotheseattributesbeingnotasflexibleasregulardocumentattributes.Forexample,the _fromand _toattributevaluesof
anexistingedgecouldnotbeupdatedoncetheedgewascreated.Additionally,the _fromand _toattributescouldnotbeindexedin
user-definedindexes,e.g.tomakeeachcombinationof _fromand _tounique.Finally,as _fromand _toreferencedthelinked
collectionsbycollectionidandnotbycollectionname,theirmeaningbecameunclearonceareferencedcollectionwasdropped.The
collectionidstoredinedgesthenbecameunusable,andwhenaccessingsuchedgethecollectionnamepartofitwasalwaystranslatedto
_undefined.
InArangoDB3.0,the _fromand _tovaluesofedgesaresavedasregularstrings.Thisallowsusing _fromand _toinuser-defined
indexes.Additionallythisallowsupdatingthe _fromand _tovaluesofexistingedges.Furthermore,collectionsreferencedby _from
and _tovaluesmaybedroppedandre-createdlater.Any _fromand _tovaluesofedgespointingtosuchdroppedcollectionare
unaffectedbythedropoperationnow.Alsonotethatrenamingthecollectionreferencedin _fromand _toinArangoDB2.8also
relinkedtheedges.In3.0theedgesareNOTautomaticallyrelinkedtothenewcollectionanymore.
Documents
Documents(incontrasttoedges)cannotcontaintheattributes _fromor _toonthemainlevelinArangoDB3.0.Theseattributeswill
beautomaticallyremovedwhensavingdocuments(i.e.non-edges). _fromand _tocanbestillusedinsub-objectsinsidedocuments.
The _fromand _toattributeswillofcoursebepreservedandarestillrequiredwhensavingedges.
AQL
Edgeshandling
WhenupdatingorreplacingedgesviaAQL,anymodificationstothe _fromand _toattributesofedgeswereignoredbyprevious
versionsofArangoDB,withoutsignalinganyerrors.Thiswasduetothe _fromand _toattributesbeingimmutableinearlierversions
ofArangoDB.
From3.0on,the _fromand _toattributesofedgesaremutable,soanyAQLqueriesthatmodifythe _fromor _toattributevalues
ofedgeswillattempttoactuallychangetheseattributes.Clientsshouldbeawareofthischangeandshouldreviewtheirqueriesthat
modifyedgestoruleoutunintendedside-effects.
Additionally,whencompletelyreplacingthedataofexistingedgesviatheAQL REPLACEoperation,itisnowrequiredtospecifyvalues
forthe _fromand _toattributes,as REPLACErequirestheentirenewdocumenttobespecified.Ifeither _fromor _toaremissing
fromthereplacementdocument,an REPLACEoperationwillfail.
Graphfunctions
Inversion3.0allformergraphrelatedfunctionshavebeenremovedfromAQLtobereplacedbynativeAQLconstructs.These
constructsallowformorefine-grainedfilteringonseveralgraphlevels.AlsothisallowstheAQLoptimizertoautomaticallyimprove
thesequeriesbyenhancingthemwithappropriateindexes.Wehavecreatedrecipestoupgradefrom2.8to3.0whenusingthese
functions.
Thefunctions:
GRAPH_COM M ON_NEIGHBORS
GRAPH_COM M ON_PROPERTIES
GRAPH_DISTANCE_TO
GRAPH_EDGES
GRAPH_NEIGHBORS
GRAPH_TRAVERSAL
GRAPH_TRAVERSAL_TREE
GRAPH_SHORTEST_PATH
GRAPH_PATHS
GRAPH_VERTICES
487
Incompatiblechangesin3.0
arecoveredinM igratingGRAPH_*Functionsfrom2.8orearlierto3.0
GRAPH_ABSOLUTE_BETWEENNESS
GRAPH_ABSOLUTE_CLOSENESS
GRAPH_ABSOLUTE_ECCENTRICITY
GRAPH_BETWEENNESS
GRAPH_CLOSENESS
GRAPH_DIAM ETER
GRAPH_ECCENTRICITY
GRAPH_RADIUS
arecoveredinM igratingGRAPH_*M easurementsfrom2.8orearlierto3.0
EDGES
NEIGHBORS
PATHS
TRAVERSAL
TRAVERSAL_TREE
arecoveredinM igratinganonymousgraphfunctionsfrom2.8orearlierto3.0
Typecastingfunctions
Thetypecastingappliedbythe TO_NUMBER()AQLfunctionhaschangedasfollows:
stringvaluesthatdonotcontainavalidnumericvaluearenowconvertedtothenumber 0.InpreviousversionsofArangoDBsuch
stringvalueswereconvertedtothevalue null.
arrayvalueswithmorethan1memberarenowconvertedtothenumber 0.InpreviousversionsofArangoDBsucharrayswere
convertedtothevalue null.
objects/documentsarenowconvertedtothenumber 0.InpreviousversionsofArangoDBobjects/documentswereconvertedto
thevalue null.
Additionally,the TO_STRING()AQLfunctionnowconverts nullvaluesintoanemptystring( "")insteadofthestring "null",
whichismoreinlinewith LENGTH(null)returning 0andnot 4sincev2.6.
Theoutputof TO_STRING()hasalsochangedforarraysandobjectsasfollows:
arraysarenowconvertedintotheirJSON-stringifyequivalents,e.g.
[]isnowconvertedto []
[1,2,3]isnowconvertedto [1,2,3]
["test",1,2]isnowconvertedto["test",1,2]`
PreviousversionsofArangoDBconvertedarrayswithnomembersintotheemptystring,andnon-emptyarraysintoacommaseparatedlistofmembervalues,withoutthesurroundingangularbrackets.Additionally,stringarraymemberswerenotenclosedin
quotesintheresultstring:
[]wasconvertedto``
[1,2,3]wasconvertedto 1,2,3
["test",1,2]wasconvertedtotest,1,2`
objectsarenowconvertedtotheirJSON-stringifyequivalents,e.g.
{}isconvertedto {}
{a:1,b:2}isconvertedto {"a":1,"b":2}
{"test":"foobar"}isconvertedto {"test":"foobar"}
PreviousversionsofArangoDBalwaysconvertedobjectsintothestring [objectObject]
ThischangealsoaffectsotherpartsinAQLthatused TO_STRING()toimplicitlycastoperandstostrings.ItalsoaffectstheAQL
functions CONCAT()and CONCAT_SEPARATOR()whichtreatedarrayvaluesdifferently.PreviousversionsofArangoDBautomatically
flattenedarrayvaluesinthefirstlevelofthearray,e.g. CONCAT([1,2,3,[4,5,6]])produced 1,2,3,4,5,6.Nowthiswillproduce
[1,2,3,[4,5,6]].Toflattenarraymembersonthetoplevel,youcannowusethemoreexplicit CONCAT(FLATTEN([1,2,3,[4,5,6]],
1)).
488
Incompatiblechangesin3.0
Arithmeticoperators
AsthearithmeticoperationsinAQLimplicitlyconverttheiroperandstonumericvaluesusing TO_NUMBER(),theircastingbehaviorhas
alsochangedasdescribedabove.
Someexamplesofthechangedbehavior:
"foo"+1produces 1now.Inpreviousversionsthisproduced null.
[1,2]+1produces 1.Inpreviousversionsthisproduced null.
1+"foo"+1´produces2 now.Inpreviousversionthisproduced1`.
Attributenamesandparameters
PreviousversionsofArangoDBhadsometroublewithattributenamesthatcontainedthedotsymbol( .).SomecodepartsinAQL
usedthedotsymboltosplitanattributenameintosub-components,soanattributenamed a.bwasnotcompletelydistinguishable
fromanattribute awithasub-attribute b.Thisinconsistentbehaviorsometimesallowed"hacks"toworksuchaspassingsubattributesinabindparameterasfollows:
FORdocINcollection
FILTERdoc.@name==1
RETURNdoc
Ifthebindparameter @namecontainedthedotsymbol(e.g. @bind= a.b,itwasunclearwhetherthisshouldtriggersub-attribute
access(i.e. doc.a.b)oraaccesstoanattributewithexactlythespecifiedname(i.e. doc["a.b"]).
ArangoDB3.0nowhandlesattributenamescontainingthedotsymbolproperly,andsendingabindparameter @name= a.bwillnow
alwaystriggeranaccesstotheattribute doc["a.b"],notthesub-attribute bof ain doc.
Forusersthatusedthe"hack"ofpassingbindparameterscontainingdotsymboltoaccesssub-attributes,ArangoDB3.0allows
specifyingtheattributenamepartsasanarrayofstrings,e.g. @name= ["a","b"],whichwillberesolvedtothesub-attributeaccess
doc.a.bwhenthequeryisexecuted.
Keywords
LIKEisnowakeywordinAQL.Using LIKEineithercaseasanattributeorcollectionnameinAQLqueriesnowrequiresquoting.
SHORTEST_PATHisnowakeywordinAQL.Using SHORTEST_PATHineithercaseasanattributeorcollectionnameinAQLqueriesnow
requiresquoting.
Subqueries
Queriesthatcontainsubqueriesthatcontaindata-modificationoperationssuchas INSERT, UPDATE, REPLACE, UPSERTor REMOVE
willnowrefusetoexecuteifthecollectionaffectedbythesubquery'sdata-modificationoperationisread-accessedinanouterscopeof
thequery.
Forexample,thefollowingquerywillrefusetoexecuteasthecollection myCollectionismodifiedinthesubquerybutalsoreadaccessedintheouterscope:
FORdocINmyCollection
LETchanges=(
FORwhatINmyCollection
FILTERwhat.value==1
REMOVEwhatINmyCollection
)
RETURNdoc
Itisstillpossibletowritetocollectionsfromwhichdataisreadinthesamequery,e.g.
FORdocINmyCollection
FILTERdoc.value==1
REMOVEdocINmyCollection
489
Incompatiblechangesin3.0
andtomodifydataindifferentcollectionviasubqueries.
Otherchanges
TheAQLoptimizerrule"merge-traversal-filter"thatalreadyexistedin3.0wasrenamedto"optimize-traversals".Thisshouldbeofno
relevancetoclientapplicationsexceptiftheyprogramaticallylookforappliedoptimizerrulesintheexplainoutofAQLqueries.
TheorderofresultscreatedbytheAQLfunctions VALUES()and ATTRIBUTES()wasneverguaranteedanditonlyhadthe"correct"
orderingbyaccidentwheniteratingoverobjectsthatwerenotloadedfromthedatabase.Assomeofthefunctioninternalshavechanged,
the"correct"orderingwillnotappearanymore,andstillnoresultorderisguaranteedbythesefunctionsunlessthe sortparameteris
specified(forthe ATTRIBUTES()function).
UpgradedV8version
TheV8enginethatisusedinsideArangoDBtoexecuteJavaScriptcodehasbeenupgradedfromversion4.3.61to5.0.71.39.Thenew
versionshouldbemostlycompatibletotheoldversion,buttheremaybesubtledifferences,includingchangesoferrormessagetexts
thrownbytheengine.Furthermore,someV8startupparametershavechangedtheirmeaningorhavebeenremovedinthenewversion.
ThisisonlyrelevantwhenArangoDBorArangoShellarestartedwithacustomvalueforthe --javascript.v8-optionsstartupoption.
Amongothers,thefollowingV8optionschangeinthenewversionofArangoDB:
--es_staging:in2.8ithadthemeaning enableallcompletedharmonyfeatures,in3.0theoptionmeans enabletest-worthy
harmonyfeatures(forinternaluseonly)
--strong_this:thisoptionwasn'tpresentin2.8.In3.0itmeans don'tallow'this'toescapefromconstructorsanddefaultsto
true.
--harmony_regexps:thisoptionsmeans enable"harmonyregularexpressionextensions"andchangesitsdefaultvaluefromfalse
totrue
--harmony_proxies:thisoptionsmeans enable"harmonyproxies"andchangesitsdefaultvaluefromfalsetotrue
--harmony_reflect:thisoptionsmeans enable"harmonyReflectAPI"andchangesitsdefaultvaluefromfalsetotrue
--harmony_sloppy:thisoptionsmeans enable"harmonyfeaturesinsloppymode"andchangesitsdefaultvaluefromfalsetotrue
--harmony_tostring:thisoptionsmeans enable"harmonytoString"andchangesitsdefaultvaluefromfalsetotrue
--harmony_unicode_regexps:thisoptionsmeans enable"harmonyunicoderegexps"andchangesitsdefaultvaluefromfalsetotrue
--harmony_arrays, --harmony_array_includes, --harmony_computed_property_names, --harmony_arrow_functions, -harmony_rest_parameters, --harmony_classes, --harmony_object_literals, --harmony_numeric_literals, --harmony_unicode:
theseoptionhavebeenremovedinV85.
AsaconsequenceoftheupgradetoV8version5,theimplementationoftheJavaScript Bufferobjecthadtobechanged.JavaScript
BufferobjectsinArangoDBnowalwaysstoretheirdataontheheap.ThereisnosharedpoolforsmallBuffervalues,andnopointing
intoexistingBufferdatawhenextractingslices.ThischangemayincreasethecostofcreatingBufferswithshortcontentsorwhen
peekingintoexistingBuffers,butwasrequiredforsafermemorymanagementandtopreventleaks.
JavaScriptAPIchanges
ThefollowingincompatiblechangeshavebeenmadetotheJavaScriptAPIinArangoDB3.0:
Foxx
TheFoxxframeworkhasbeencompletelyrewrittenfor3.0withanew,simplerandmorefamiliarAPI.TomakeFoxxservicesdeveloped
for2.8orearlierArangoDBversionsrunin3.0,theservice'smanifestfileneedstobeedited.
ToenablethelegacymodeforaFoxxservice,add "engines":{"arangodb":"^2.8.0"}(orsimilarversionrangesthatexclude3.0andup)
totheservicemanifestfile(named"manifest.json",locatedintheservice'sbasedirectory).
490
Incompatiblechangesin3.0
Require
M odulesshippedwithArangoDBcannowberequiredusingthepattern @arangodb/<module>insteadof org/arangodb/<module>,e.g.
varcluster=require("@arangodb/cluster");
Theoldformatcanstillbeusedforcompatibility:
varcluster=require("org/arangodb/cluster");
ArangoDBpriortoversion3.0allowedatransparentuseofCoffeeScriptsourcefileswiththe require()function.Fileswithafile
nameextensionof coffeewereautomaticallysentthroughaCoffeeScriptparserandtranspiledintoJavaScripton-the-fly.Thissupport
isgonewithArangoDB3.0.TorunanyCoffeeScriptsourcefiles,theymustbeconvertedtoJavaScriptbytheclientapplication.
Responseobject
The @arangodb/requestresponseobjectnowstorestheparsedJSONresponsebodyinaproperty jsoninsteadof bodywhenthe
requestwasmadeusingthe jsonoption.The bodyinsteadcontainstheresponsebodyasastring.
EdgesAPI
Whencompletelyreplacinganedgeviaacollection's replace()functionthereplacingedgedatanowneedstocontainthe _fromand
_toattributesforthenewedge.PreviousversionsofArangoDBdidnotrequiretheedgedatatocontain _fromand _toattributes
whenreplacinganedge,since _fromand _tovalueswereimmutableforexistingedges.
Forexample,thefollowingcallworkedinArangoDB2.8butwillfailin3.0:
db.edgeCollection.replace("myKey",{value:"test"});
TomakethisworkinArangoDB3.0, _fromand _toneedtobeaddedtothereplacementdata:
db.edgeCollection.replace("myKey",{_from:"myVertexCollection/1",_to:"myVertexCollection/2",value:"test"});
Notethatthisonlyaffectsthe replace()functionbutnot update(),whichwillonlyupdatethespecifiedattributesoftheedgeand
leaveallothersintact.
Additionally,thefunctions edges(), outEdges()and inEdges()withanarrayofedgeidswillnowmaketheedgeidsuniquebefore
returningtheconnectededges.Thisisprobablydesiredanyway,asresultswillbereturnedonlyonceperdistinctinputedgeid.However,
itmaybreakclientapplicationsthatrelyontheoldbehavior.
DatabasesAPI
The _listDatabases()functionofthe dbobjecthasbeenrenamedto _databases(),makingitconsistentwiththe _collections()
function.Alsothe _listEndpoints()functionhasbeenrenamedto _endpoints().
CollectionAPI
Examplematching
Thecollectionfunction byExampleHash()and byExampleSkiplist()havebeenremovedin3.0.Theirfunctionalityisprovidedby
collection's byExample()function,whichwillautomaticallyuseasuitableindexifpresent.
Thecollectionfunction byConditionSkiplist()hasbeenremovedin3.0.ThesamefunctionalitycanbeachievedbyissuinganAQL
querywiththetargetcondition,whichwillautomaticallyuseasuitableindexifpresent.
Revisionidhandling
491
Incompatiblechangesin3.0
The exists()methodofacollectionnowthrowsanexceptionwhenthespecifieddocumentexistsbutitsrevisioniddoesnotmatch
therevisionidspecified.PreviousversionsofArangoDBsimplyreturned falseifeithernodocumentexistedwiththespecifiedkeyor
whentherevisioniddidnotmatch.Itwasthereforeimpossibletodistinguishthesetwocasesfromthereturnvaluealone.3.0corrects
this.Additionally, exists()inpreviousversionsalwaysreturnedabooleanifonlythedocumentkeywasgiven.3.0nowreturnsthe
document'smeta-data,whichincludesthedocument'scurrentrevisionid.
Giventhereisadocumentwithkey testincollection myCollection,thenthebehaviorof3.0isasfollows:
/*testifdocumentexists.thisreturnedtruein2.8*/
db.myCollection.exists("test");
{
"_key":"test",
"_id":"myCollection/test",
"_rev":"9758059"
}
/*testifdocumentexists.thisreturnedtruein2.8*/
db.myCollection.exists({_key:"test"});
{
"_key":"test",
"_id":"myCollection/test",
"_rev":"9758059"
}
/*testifdocumentexists.thisalsoreturnedfalsein2.8*/
db.myCollection.exists("foo");
false
/*testifdocumentwithagivenrevisionidexists.thisreturnedtruein2.8*/
db.myCollection.exists({_key:"test",_rev:"9758059"});
{
"_key":"test",
"_id":"myCollection/test",
"_rev":"9758059"
}
/*testifdocumentwithagivenrevisionidexists.thisreturnedfalsein2.8*/
db.myCollection.exists({_key:"test",_rev:"1234"});
JavaScriptexception:ArangoError1200:conflict
Capconstraints
Thecapconstraintsfeaturehasbeenremoved.Thischangehasledtotheremovalofthecollectionoperations first()and last(),
whichwereinternallybasedondatafromcapconstraints.
AscapconstraintshavebeenremovedinArangoDB3.0itisnotpossibletocreateanindexoftype"cap"withacollection's
ensureIndex()function.Thededicatedfunction ensureCapConstraint()hasalsobeenremovedfromthecollectionAPI.
GraphBlueprintsJSModule
Thedeprecatedmodule graph-blueprintshasbeendeleted.Allit'sfeaturesarecoveredbythe general-graphmodule.
GeneralGraphFluentAQLinterface
ThefluentinterfacehasbeenremovedfromArangoDB.It'sfeatureswerecompletelyoverlappingwith"aqb"whichcomespreinstalled
aswell.PleaseswitchtoAQBinstead.
UndocumentedAPIs
Theundocumentedfunctions BY_EXAMPLE_HASH()and BY_EXAMPLE_SKIPLIST(), BY_CONDITION_SKIPLIST, CPP_NEIGHBORSand
CPP_SHORTEST_PATHhavebeenremoved.ThesefunctionswerealwayshiddenandnotintendedtobepartofthepublicJavaScriptAPI
forcollections.
HTTPAPIchanges
492
Incompatiblechangesin3.0
CRUDoperations
ThefollowingincompatiblechangeshavebeenmadetotheHTTPAPIinArangoDB3.0:
General
TheHTTPinsertoperationsforsingledocumentsandedges(POST /_api/document)donotsupporttheURLparameter
"createCollection"anymore.InpreviousversionsofArangoDBthisparametercouldbeusedtoautomaticallycreateacollectionupon
insertionofthefirstdocument.ItisnowrequiredthatthetargetcollectionalreadyexistswhenusingthisAPI,otherwiseitwillreturnan
HTTP404error.ThesameistruefortheimportAPIatPOST /_api/import.
CollectionscanstillbecreatedeasilyviaaseparatecalltoPOST /_api/collectionasbefore.
The"location"HTTPheaderreturnedbyArangoDBwheninsertinganewdocumentoredgenowalwayscontainsthedatabasename.
ThiswasalsothedefaultbehaviorinpreviousversionsofArangoDB,butitcouldbeoverriddenbyclientssendingtheHTTPheader xarango-version:1.4intherequest.ClientscancontinuetosendthisheadertoArangoDB3.0,buttheheaderwillnotinfluencethe
locationresponseheadersproducedbyArangoDB3.0anymore.
AdditionallytheCRUDoperationsAPIsdonotreturnanattribute"error"intheresponsebodywithanattributevalueof"false"incase
anoperationsucceeded.
Revisionidhandling
Theoperationsforupdating,replacingandremovingdocumentscanoptionallychecktherevisionnumberofthedocumenttobeupdated,
replacedorremovedsothecallercanensuretheoperationworksonaspecificversionofthedocumentandtherearenolostupdates.
PreviousversionsofArangoDBallowedpassingtherevisionidofthepreviousdocumenteitherintheHTTPheader If-Matchorinthe
URLparameter rev.Forexample,removingadocumentwithaspecificrevisionidcouldbeachievedasfollows:
curl-XDELETE\
"http://127.0.0.1:8529/_api/document/myCollection/myKey?rev=123"
ArangoDB3.0doesnotsupportpassingtherevisionidviathe"rev"URLparameteranymore.Insteadthepreviousrevisionidmustbe
passedintheHTTPheader If-Match,e.g.
curl-XDELETE\
--header"If-Match:'123'"\
"http://127.0.0.1:8529/_api/document/myCollection/myKey"
TheURLparameter"policy"wasalsousableinpreviousversionsofArangoDBtocontrolrevisionhandling.Usingitwasredundantto
specifyingtheexpectedrevisionidviathe"rev"parameteror"If-M atch"HTTPheaderandthereforesupportforthe"policy"parameter
wasremovedin3.0.
Inordertocheckforapreviousrevisionidwhenupdating,replacingorremovingdocumentspleaseusethe If-MatchHTTPheaderas
describedabove.WhennorevisioncheckifrequiredtheHTTPheadercanbeomitted,andtheoperationswillworkonthecurrent
revisionofthedocument,regardlessofitsrevisionid.
AlldocumentsAPI
TheHTTPAPIforretrievingtheids,keysorURLsofalldocumentsfromacollectionwaspreviouslylocatedatGET /_api/document?
collection=....ThisAPIwasmovedtoPUT /_api/simple/all-keysandisnowexecutedasanAQLquery.Thenameofthe
collectionmustnowbepassedintheHTTPrequestbodyinsteadofintherequestURL.Thesameistrueforthe"type"parameter,
whichcontrolsthetypeoftheresulttobecreated.
CallstothepreviousAPIcanbetranslatedasfollows:
old:GET /_api/document?collection=<collection>&type=<type>withoutHTTPrequestbody
3.0:PUT /_api/simple/all-keyswithHTTPrequestbody {"collection":"<collection>","type":"id"}
TheresultformatofthisAPIhasalsochangedslightly.InpreviousversionscallstotheAPIreturnedaJSONobjectwitha documents
attribute.AsthefunctionalityisbasedonAQLinternallyin3.0,theAPInowreturnsaJSONobjectwitha resultattribute.
493
Incompatiblechangesin3.0
EdgesAPI
CRUDoperations
TheAPIfordocumentsandedgeshavebeenunifiedinArangoDB3.0.TheCRUDoperationsfordocumentsandedgesarenowhandled
bythesameendpointat /_api/document.ForCRUDoperationsthereisnodistinctionanymorebetweendocumentsandedgesAPIwise.
ThatmeansCRUDoperationsconcerningedgesneedtobesenttotheHTTPendpoint /_api/documentinsteadof /_api/edge.Sending
requeststo /_api/edgewillresultinanHTTP404errorin3.0.Thefollowingmethodsareavailableat /_api/documentfordocuments
andedge:
HTTPPOST:insertnewdocumentoredge
HTTPGET:fetchanexistingdocumentoredge
HTTPPUT:replaceanexistingdocumentoredge
HTTPPATCH:partiallyupdateanexistingdocumentoredge
HTTPDELETE:removeanexistingdocumentoredge
WhencompletelyreplacinganedgeviaHTTPPUTpleasenotethatthereplacingedgedatanowneedstocontainthe _fromand _to
attributesfortheedge.PreviousversionsofArangoDBdidnotrequiresending _fromand _towhenreplacingedges,as _fromand
_tovalueswereimmutableforexistingedges.
The _fromand _toattributesofedgesnowalsoneedtobepresentinsidetheedgesobjectssenttotheserver:
curl-XPOST\
--data'{"value":1,"_from":"myVertexCollection/1","_to":"myVertexCollection/2"}'\
"http://127.0.0.1:8529/_api/document?collection=myEdgeCollection"
PreviousversionsofArangoDBrequiredthe _fromand _toattributesofedgesbesentseparatelyinURLparameter fromand to:
curl-XPOST\
--data'{"value":1}'\
"http://127.0.0.1:8529/_api/edge?collection=e&from=myVertexCollection/1&to=myVertexCollection/2"
Queryingconnectededges
TheRESTAPIforqueryingconnectededgesatGET /_api/edges/<collection>willnowmaketheedgeidsuniquebeforereturningthe
connectededges.Thisisprobablydesiredanywayasresultswillnowbereturnedonlyonceperdistinctinputedgeid.However,itmay
breakclientapplicationsthatrelyontheoldbehavior.
GraphAPI
Somedata-modificationoperationsinthenamedgraphsAPIat /_api/gharialnowreturneitherHTTP202(Accepted)orHTTP201
(Created)iftheoperationsucceeds.Whichstatuscodeisreturneddependsonthe waitForSyncattributeoftheaffectedcollection.In
previousversionssomeoftheseoperationsreturnHTTP200regardlessofthe waitForSyncvalue.
ThedeprecatedgraphAPI /_api/graphhasbeenremoved.Allit'sfeaturescanbereplacedusing /_api/gharialandAQLinstead.
SimplequeriesAPI
TheRESTroutesPUT /_api/simple/firstand /_api/simple/lasthavebeenremovedentirely.TheseAPIswereresponsiblefor
returningthefirst-insertedandlast-inserteddocumentsinacollection.Thisfeaturewasbuiltoncapconstraintsinternally,whichhave
beenremovedin3.0.
Callingoneoftheseendpointsin3.0willresultinanHTTP404error.
IndexesAPI
494
Incompatiblechangesin3.0
Itisnotsupportedin3.0tocreateanindexwithtype cap(capconstraint)in3.0asthecapconstraintsfeaturehasbeeremoved.
CallingtheindexcreationendpointHTTPAPIPOST /_api/index?collection=...withanindextype capwillthereforeresultinan
HTTP400error.
LogentriesAPI
TheRESTrouteHTTPGET /_admin/logisnowaccessiblefromwithinalldatabases.InpreviousversionsofArangoDB,thisroute
wasaccessiblefromwithinthe _systemdatabaseonly,andanHTTP403(Forbidden)wasthrownbytheserverforanyaccessfrom
withinanotherdatabase.
FiguresAPI
TheRESTrouteHTTPGET /_api/collection/<collection>/figureswillnotreturnthefollowingresultattributesastheybecame
meaninglessin3.0:
shapefiles.count
shapes.fileSize
shapes.count
shapes.size
attributes.count
attributes.size
DatabasesandCollectionsAPIs
WhencreatingadatabaseviatheAPIPOST /_api/database,ArangoDBwillnowalwaysreturntheHTTPstatuscode202(created)if
theoperationsucceeds.PreviousversionsofArangoDBreturnedHTTP202aswell,butthisbehaviorwaschangablebysendingan
HTTPheader x-arango-version:1.4.Whensendingthisheader,previousversionsofArangoDBreturnedanHTTPstatuscode200
(ok).ClientscanstillsendthisheadertoArangoDB3.0butthiswillnotinfluencetheHTTPstatuscodeproducedbyArangoDB.
The"location"headerproducedbyArangoDB3.0willnowalwayscontainthedatabasename.Thiswasalsothedefaultinprevious
versionsofArangoDB,butthebehaviorcouldbeoverriddenbysendingtheHTTPheader x-arango-version:1.4.Clientscanstillsend
theheader,butthiswillnotmakethedatabasenameinthe"location"responseheaderdisappear.
TheresultformatforqueryingallcollectionsviatheAPIGET /_api/collectionhasbeenchanged.
PreviousversionsofArangoDBreturnedanobjectwithanattributenamed collectionsandanattributenamed names.Bothcontained
allavailablecollections,but collectionscontainedthecollectionsasanarray,and namescontainedthecollectionsagain,containedin
anobjectinwhichtheattributenameswerethecollectionnames,e.g.
{
"collections":[
{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
...
],
"names":{
"test":{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
"something":{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
...
}
}
Thisresultstructurewasredundant,andthereforehasbeensimplifiedtojust
{
"result":[
{"id":"5874437","name":"test","isSystem":false,"status":3,"type":2},
{"id":"17343237","name":"something","isSystem":false,"status":3,"type":2},
...
]
}
inArangoDB3.0.
495
Incompatiblechangesin3.0
ReplicationAPIs
TheURLparameter"failOnUnknown"wasremovedfromtheRESTAPIGET /_api/replication/dump.Thisparametercontrolled
whetherdumpingorreplicatingedgesshouldfailifoneofthevertexcollectionslinkedintheedge's _fromor _toattributeswasnot
presentanymore.Inthiscasethe _fromand _tovaluescouldnotbetranslatedintomeaningfulidsanymore.
Thereweretwowaysforhandlingthis:
setting failOnUnknownto truecausedtheHTTPrequesttofail,leavingerrorhandlingtotheuser
setting failOnUnknownto falsecausedtheHTTPrequesttocontinue,translatingthecollectionnamepartinthe _fromor _to
valueto _unknown.
InArangoDB3.0thisparameterisobsolete,as _fromand _toarestoredasself-containedstringvaluesallthetime,sotheycannot
getinvalidwhenreferencedcollectionsaredropped.
TheresultformatoftheAPIGET /_api/replication/logger-followhaschangedslightlyinthefollowingaspects:
documentsandedgesarereportedinthesameway.Thetypefordocumentinsertions/updatesandedgeinsertions/updatesisnow
always 2300.PreviousversionsofArangoDBreturneda typevalueof 2300fordocumentsand 2301foredges.
recordsaboutinsertions,updatesorremovalsofdocumentsandedgesdonothavethe keyand revattributesonthetop-level
anymore.Instead, keyand revcanbeaccessedbypeekingintothe _keyand _revattributesofthe datasub-attributesof
thechangerecord.
Thesameistrueforthecollection-specificchangesAPIGET /_api/replication/dump.
UsermanagementAPIs
TheRESTAPIendpointPOST /_api/userforaddingnewusersnowrequirestherequesttocontainaJSONobjectwithanattribute
named user,containingthenameoftheusertobecreated.PreviousversionsofArangoDBalsocheckedthisattribute,butadditionally
lookedforanattribute usernameifthe userattributedidnotexist.
UndocumentedAPIs
ThefollowingundocumentedHTTPRESTendpointshavebeenremovedfromArangoDB'sRESTAPI:
/_open/cerberusand /_system/cerberus:theseendpointswereintendedforsomeArangoDB-internalapplicationsonly
PUT /_api/simple/by-example-hash,PUT /_api/simple/by-example-skiplistandPUT /_api/simple/by-condition-skiplist:
thesemethodsweredocumentedinearlyversionsofArangoDBbuthavebeenmarkedasnotintendedtobecalledbyendusers
sinceArangoDBversion2.3.ThesemethodsshouldnothavebeenpartofanyArangoDBmanualsinceversion2.4.
/_api/structure:anolderunfinishedandunpromotedAPIfordataformatandtypechecks,supersededbyFoxxapplications.
AdministrationAPIs
/_admin/shutdownnowneedstobecalledwiththeHTTPDELETEmethod
HandlingofCORSrequests
ItcannowbecontrolledindetailforwhichoriginhostsCORS(Cross-originresourcesharing)requestswithcredentialswillbeallowed.
ArangoDB3.0providesthestartupoption --http.trusted-originthatcanbeusedtospecifyoneormanyoriginsfromwhichCORS
requestsaretreatedas"trustworthy".
Theoptioncanbespecifiedmultipletimes,oncepertrustedorigin,e.g.
--http.trusted-originhttp://127.0.0.1:8529--http.trusted-originhttps://127.0.0.1:8599
ThiswillmaketheArangoDBserverrespondtoCORSrequestsfromtheseoriginswithan Access-Control-Allow-CredentialsHTTP
headerwithavalueof true.WebbrowserscaninspectthisheaderandcanallowpassingArangoDBwebinterfacecredentials(ifstored
inthebrowser)totherequestingsite.ArangoDBwillnotforwardorprovideanycredentials.
496
Incompatiblechangesin3.0
SettingthisoptionisonlyrequiredifapplicationsonotherhostsneedtoaccesstheArangoDBwebinterfaceorotherHTTPRESTAPIs
fromawebbrowserwiththesamecredentialsthattheuserhasenteredwhenloggingintothewebinterface.Whenawebbrowserfinds
the Access-Control-Allow-CredentialsHTTPresponseheader,itmayforwardthecredentialsenteredintothebrowserforthe
ArangoDBwebinterfacelogintotheothersite.
Thisisapotentialsecurityissue,sotherearenotrustedoriginsbydefault.Itmayberequiredtosetsometrustedoriginsifyou're
planningtoissueAJAXrequeststoArangoDBfromothersitesfromthebrowser,withthecredentialsenteredduringtheArangoDB
interfacelogin(i.e.singlesign-on).Ifsuchfunctionalityisnotused,theoptionshouldnotbeset.
Tospecifyatrustedorigin,specifytheoptiononcepertrustedoriginasshownabove.Notethatthetrustedoriginvaluesspecifiedin
thisoptionwillbecomparedbytewisewiththe OriginHTTPheadervaluesentbyclients,andonlyexactmatcheswillpass.
Thereisalsothewildcard allforenablingCORSaccessfromalloriginsinatestordevelopmentsetup:
--http.trusted-originall
SettingthisoptionwillleadtotheArangoDBserverrespondingwithan Access-Control-Allow-Credentials:trueHTTPheadertoall
incomingCORSrequests.
Command-lineoptions
QuiteafewstartupoptionsinArangoDB2weredoublenegations(like --server.disable-authenticationfalse).InArangoDB3these
arenowexpressedaspositives(e.g. --server.authentication).AlsotheoptionsbetweentheArangoDBserveranditsclienttoolshave
beingunified.Forexample,theloggeroptionsarenowthesamefortheserverandtheclienttools.Additionallymanyoptionshavebeen
movedintomoreappropriatetopicsections.
Renamedoptions
Thefollowingoptionshavebeenavailablebefore3.0andhavechangedtheirnamein3.0:
--server.disable-authenticationwasrenamedto --server.authentication.Notethatthemeaningoftheoption -server.authenticationistheoppositeoftheprevious --server.disable-authentication.
--server.disable-authentication-unix-socketswasrenamedto --server.authentication-unix-sockets.Notethatthemeaningof
theoption --server.authentication-unix-socketsistheoppositeoftheprevious --server.disable-authentication-unix-sockets.
--server.authenticate-system-onlywasrenamedto --server.authentication-system-only.Themeaningoftheoptionin
unchanged.
--server.disable-statisticswasrenamedto --server.statistics.Notethatthemeaningoftheoption --server.statisticsis
theoppositeoftheprevious --server.disable-statistics.
--server.cafilewasrenamedto --ssl.cafile.Themeaningoftheoptionisunchanged.
--server.keyfilewasrenamedto --ssl.keyfile.Themeaningoftheoptionisunchanged.
--server.ssl-cachewasrenamedto --ssl.session-cache.Themeaningoftheoptionisunchanged.
--server.ssl-cipher-listwasrenamedto --ssl.cipher-list.Themeaningoftheoptionisunchanged.
--server.ssl-optionswasrenamedto --ssl.options.Themeaningoftheoptionisunchanged.
--server.ssl-protocolwasrenamedto --ssl.protocol.Themeaningoftheoptionisunchanged.
--server.backlog-sizewasrenamedto --tcp.backlog-size.Themeaningoftheoptionisunchanged.
--server.reuse-addresswasrenamedto --tcp.reuse-address.Themeaningoftheoptionisunchanged.
--server.disable-replication-applierwasrenamedto --database.replication-applier.Themeaningoftheoption -database.replication-applieristheoppositeoftheprevious --server.disable-replication-applier.
--server.allow-method-overridewasrenamedto --http.allow-method-override.Themeaningoftheoptionisunchanged.
--server.hide-product-headerwasrenamedto --http.hide-product-header.Themeaningoftheoptionisunchanged.
--server.keep-alive-timeoutwasrenamedto --http.keep-alive-timeout.Themeaningoftheoptionisunchanged.
--server.foxx-queueswasrenamedto --foxx.queues.Themeaningoftheoptionisunchanged.
--server.foxx-queues-poll-intervalwasrenamedto --foxx.queues-poll-interval.Themeaningoftheoptionisunchanged.
--no-serverwasrenamedto --server.rest-server.Notethatthemeaningoftheoption --server.rest-serveristheopposite
oftheprevious --no-server.
--database.query-cache-modewasrenamedto --query.cache-mode.Themeaningoftheoptionisunchanged.
--database.query-cache-max-resultswasrenamedto --query.cache-entries.Themeaningoftheoptionisunchanged.
497
Incompatiblechangesin3.0
--database.disable-query-trackingwasrenamedto --query.tracking.Themeaningoftheoption --query.trackingisthe
oppositeoftheprevious --database.disable-query-tracking.
--log.ttywasrenamedto --log.foreground-tty.Themeaningoftheoptionisunchanged.
--upgradehasbeenrenamedto --database.auto-upgrade.Incontrastto2.8thisoptionnowrequiresabooleanparameter.To
actuallyperformanautomaticdatabaseupgradeatstartupuse --database.auto-upgradetrue.Tonotperformit,use -database.auto-upgradefalse.
--check-versionhasbeenrenamedto --database.check-version.
--temp-pathhasbeenrenamedto --temp.path.
Logverbosity,topicsandoutputfiles
Loggingnowsupportslogtopics.Youcancontrolthesebyspecifyingalogtopicinfrontofalogleveloranoutput.Forexample
--log.levelstartup=trace--log.levelinfo
willlogmessagesconcerningstartupattracelevel,everythingelseatinfolevel. --log.levelcanbespecifiedmultipletimesatstartup,
forasmanytopicsasneeded.
Somerelevantlogtopicsavailablein3.0are:
collector:informationabouttheWALcollector'sstate
compactor:informationaboutthecollectiondatafilecompactor
datafiles:datafile-relatedoperations
mmap:informationaboutmemory-mappingoperations
performance:someperformance-relatedinformation
queries:executedAQLqueries
replication:replication-relatedinfo
requests:HTTPrequests
startup:informationaboutserverstartupandshutdown
threads:informationaboutthreads
Thenewlogoption --log.output<definition>allowsdirectingtheglobalorper-topiclogoutputtodifferentoutputs.Theoutput
definition""canbeoneof
"-"forstdin
"+"forstderr
"syslog://"
"syslog:///"
"file://"
Theoptioncanbespecifiedmultipletimesinordertoconfiguretheoutputfordifferentlogtopics.Tosetupaper-topicoutput
configuration,use --log.output<topic>=<definition>,e.g.
queries=file://queries.txt
logsallqueriestothefile"queries.txt".
Theoldoption --log.fileisstillavailablein3.0forconveniencereasons.In3.0itisashortcutforthemoregeneraloption -log.outputfile://filename.
Theoldoption --log.requests-fileisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption --log.output
requests=file://....
Theoldoption --log.performanceisstillavailablein3.0.Itisnowashortcutforthemoregeneraloption --log.level
performance=trace.
Removedoptionsforlogging
Theoptions --log.content-filterand --log.source-filterhavebeenremoved.TheyhavemostbeenusedduringArangoDB's
internaldevelopment.
498
Incompatiblechangesin3.0
Thesyslog-relatedoptions --log.applicationand --log.facilityhavebeenremoved.Theyaresupersededbythemoregeneral -log.outputoptionwhichcanalsohandlesyslogtargets.
Removedotheroptions
Theoption --server.default-api-compatibilitywaspresentinearlierversionofArangoDBtocontrolvariousaspectsoftheserver
behavior,e.g.HTTPreturncodesortheformatofHTTP"location"headers.ClientapplicationscouldsendanHTTPheader"x-arangoversion"withaversionnumbertorequesttheserverbehaviorofacertainArangoDBversion.
Thisoptionwasonlyhonoredinahandfulofcases(describedabove)andwasremovedin3.0becausethechangesinserverbehavior
controlledbythisoptionwerechangedevenbeforeArangoDB2.0.Thisshouldhaveleftenoughtimeforclientapplicationstoadaptto
thenewbehavior,makingtheoptionsuperfluousin3.0.
Threadoptions
Theoptions --server.threadsand --scheduler.threadsnowhaveadefaultvalueof 0.When --server.threadsissetto 0on
startup,thesuitablenumberofthreadswillbedeterminedbyArangoDBbyaskingtheOSforthenumberofavailableCPUsandusing
thatasabaseline.IfthenumberofCPUsislowerthan4,ArangoDBwillstillstart4dispatcherthreads.When --scheduler.threadsis
setto 0,thenArangoDBwillautomaticallydeterminethenumberofschedulerthreadstostart.Thiswillnormallycreate2scheduler
threads.
Iftheexactnumberofthreadsneedstobesetbytheadmin,thenitisstillpossibletoset --server.threadsand --scheduler.threads
tonon-zerovalues.ArangoDBwillusethesevaluesandstartthatmanythreads(notethatsomethreadsmaybecreatedlazilysothey
maynotbepresentdirectlyafterstartup).
ThenumberofV8JavaScriptcontextstobecreated( --javascript.v8-contexts)nowhasadefaultvalueof 0too,meaningthat
ArangoDBwillcreateasmanyV8contextsastherewillbedispatcherthreads(controlledbythe --server.threadsoption).Settingthis
optiontoanon-zerovaluewillcreateexactlyasmanyV8contextsasspecified.
Settingtheseoptionsexplicitlytonon-zerovaluesmaybebeneficialinenvironmentsthathavefewresources(processingtime,maximum
threadcount,availablememory).
Authentication
Thedefaultvaluefor --server.authenticationisnow trueintheconfigurationfilesshippedwithArangoDB.Thismeanstheserver
willbestartedwithauthenticationenabledbydefault,requiringallclientconnectionstoprovideauthenticationdatawhenconnectingto
ArangoDBAPIs.PreviousArangoDBversionsusedthesetting --server.disable-authenticationtrue,effectivelydisabling
authenticationbydefault.
Thedefaultvaluefor --server.authentication-system-onlyisnow trueinArangoDB.ThatmeansthatFoxxapplicationsrunningin
ArangoDBwillbepublicaccessible(atleasttheywillnotuseArangoDB'sbuiltinauthenticationmechanism).Onlyrequeststo
ArangoDBAPIsatURLpathprefixes /_api/and /_adminwillrequireauthentication.Tochangethat,andusethebuiltin
authenticationmechanismforFoxxapplicationstoo,set --server.authentication-system-onlyto false,andmakesuretohavethe
option --server.authenticationsetto trueaswell.
Thoughenablingtheauthenticationisrecommendedforproductionsetups,itmaybeoverkillinadevelopmentenvironment.Toturnoff
authentication,theoption --server.authenticationcanbesetto falseinArangoDB'sconfigurationfileoronthecommand-line.
WebAdminInterface
TheJavaScriptshellhasbeenremovedfromArangoDB'swebinterface.Thefunctionalitytheshellprovidedisstillfullyavailableinthe
ArangoShell(arangosh)binaryshippedwithArangoDB.
ArangoShellandclienttools
499
Incompatiblechangesin3.0
TheArangoShell(arangosh)andtheotherclienttoolsbundledwithArangoDBcanonlyconnecttoanArangoDBserverofversion3.0or
higher.TheywillnotconnecttoanArangoDB2.8.ThisisbecausetheserverHTTPAPIshavechangedbetween2.8and3.0,andall
clienttoolsusestheseAPIs.
InordertoconnecttoearlierversionsofArangoDBwiththeclienttools,anolderversionoftheclienttoolsneedstobekeptinstalled.
Thepreferrednameforthetemplatestringgeneratorfunction aqlQueryisnow aqlandisautomaticallyavailableinarangosh.
Elsewhere,itcanbeloadedlike constaql=require('@arangodb').aql.
Command-lineoptionsadded
Allclienttoolsin3.0provideanoption --server.max-packet-sizeforcontrollingthemaximumsizeofHTTPpacketstobehandledby
theclienttools.Thedefaultvalueis128M B,asinpreviousversionsofArangoDB.Incontrasttopreviousversionsinwhichthevalue
washard-coded,theoptionisnowconfigurable.ItcanbeincreasedtomaketheclienttoolshandleverylargeHTTPresultmessagessent
bytheserver.
Command-lineoptionschanged
Forallclienttools,theoption --server.disable-authenticationwasrenamedto --server.authentication.Notethatthemeaningof
theoption --server.authenticationistheoppositeoftheprevious --server.disable-authentication.
Theoption --server.ssl-protocolwasrenamedto --ssl.protocol.Themeaningoftheoptionisunchanged.
Thecommand-lineoption --quietwasremovedfromallclienttoolsexceptarangoshbecauseithadnoeffectinthem.
Arangobench
Inordertomakeitspurposemoreapparenttheformer arangobclienttoolhasbeenrenamedto arangobenchin3.0.
Miscellaneouschanges
Thechecksumcalculationalgorithmforthe collection.checksum()methodanditscorrespondingRESTAPIGET
/_api/collection/<collection</checksumhaschangedin3.0.Checksumscalculatedin3.0willdifferfromchecksumscalculatedwith2.8
orbefore.
TheArangoDBserverin3.0doesnotreadafile ENDPOINTScontainingalistofadditionalendpointsonstartup.In2.8thisfilewas
automaticallyreadifpresentinthedatabasedirectory.
Thenamesofthesub-threadsstartedbyArangoDBhavechangedin3.0.ThisisrelevantonLinuxonly,wherethreadscanbenamedand
threadnamesmaybevisibletosystemtoolssuchastopormonitoringsolutions.
500
WhatsNewin2.8
FeaturesandImprovements
ThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.8.ArangoDB2.8alsocontainsseveral
bugfixesthatarenotlistedhere.Foralistofbugfixes,pleaseconsulttheCHANGELOG.
AQLimprovements
AQLGraphTraversals/PatternMatching
AQLoffersanewfeaturetotraverseoveragraphwithoutwritingJavaScriptfunctionsbutwithalltheotherfeaturesyouknowfrom
AQL.Forthispurpose,aspecialversionof FORvariableNameINexpressionhasbeenintroduced.
Thisspecialversionhasthefollowingformat: FORvertex-variable,edge-variable,path-variableINtraversal-expression,where
traversal-expressionhasthefollowingformat: [depth]directionstart-vertexgraph-definitionwiththefollowinginput
parameters:
depth(optional):defineshowmanystepsareexecuted.Thevaluecaneitherbeanintegervalue(e.g. 3)orarangeofintegervalues
(e.g. 1..5).Thedefaultis1.
direction:defineswhichedgedirectionsarefollowed.Canbeeither OUTBOUND, INBOUNDor ANY.
start-vertex:defineswherethetraversalisstarted.M ustbean _idvalueoradocument.
graph-definition:defineswhichedgecollectionsareusedforthetraversal.M ustbeeither GRAPHgraph-nameforgraphscreatedwith
thegraph-module,oralistofedgecollections edge-col1,edge-col2,..edge-colN.
Thethreeoutputvariableshavethefollowingsemantics:
vertex-variable:Thelastvisitedvertex.
edge-variable:Thelastvisitededge(optional).
path-variable:Thecompletepathfromstart-vertextovertex-variable(optional).
Thetraversalstatementcanbeusedinthesamewayastheoriginal FORvariableNameINexpression,andcanbecombinedwithfilters
andotherAQLconstructs.
AsanexampleonecannowfindthefriendsofafriendforacertainuserwiththisAQLstatement:
FORfoaf,e,pathIN2ANY@startUserGRAPH"relations"
FILTERpath.edges[0].type=="friend"
FILTERpath.edges[1].type=="friend"
FILTERfoaf._id!=@startUser
RETURNDISTINCTfoaf
Optimizerruleshavebeenimplementedtogainperformanceofthetraversalstatement.Theserulesmovefilterstatementsintothe
traversalstatements.t.pathswhichcanneverpassthefilterarenotemittedtothevariables.
Asanexampletakethequeryaboveandassumethereareedgesthatdonothave type=="friend".Ifinthefirstedgestepthereissuch
anon-friendedgethesecondstepswillneverbecomputedfortheseedgesastheycannotfulfillthefiltercondition.
ArrayIndexes
Hashindexesandskiplistindexescannowoptionallybedefinedforarrayvaluessothattheyindexindividualarraymembersinsteadof
theentirearrayvalue.
Todefineanindexforarrayvalues,theattributenameisextendedwiththeexpansionoperator [*]intheindexdefinition.
Example:
db._create("posts");
db.posts.ensureHashIndex("tags[*]");
Whengiventhefollowingdocument
501
WhatsNewin2.8
{
"tags":[
"AQL",
"ArangoDB",
"Index"
]
}
thisindexwillnowcontaintheindividualvalues "AQL", "ArangoDB"and "Index".
Nowtheindexcanbeusedforfindingalldocumentshaving "ArangoDB"somewhereintheir tagsarrayusingthefollowingAQL
query:
FORdocINposts
FILTER"ArangoDB"INdoc.tags[*]
RETURNdoc
Itisalsopossibletocreateanindexonsub-attributesofarrayvalues.Thismakessensewhentheindexattributeisanarrayofobjects,
e.g.
db._drop("posts");
db._create("posts");
db.posts.ensureIndex({type:"hash",fields:["tags[*].name"]});
db.posts.insert({tags:[{name:"AQL"},{name:"ArangoDB"},{name:"Index"}]});
db.posts.insert({tags:[{name:"AQL"},{name:"2.8"}]});
Thefollowingquerywillthenusethearrayindex:
FORdocINposts
FILTER'AQL'INdoc.tags[*].name
RETURNdoc
Arrayvalueswillautomaticallybede-duplicatedbeforebeinginsertedintoanarrayindex.
PleasenotethatfilteringusingarrayindexesonlyworksfromwithinAQLqueriesandonlyifthequeryfiltersontheindexedattribute
usingthe INoperator.Theothercomparisonoperators( ==, !=, >, >=, <, <=)currentlydonotusearrayindexes.
Optimizerimprovements
TheAQLqueryoptimizercannowuseindexesifmultiplefilterconditionsonattributesofthesamecollectionarecombinedwithlogical
ORs,andiftheusageofindexeswouldcompletelycovertheseconditions.
Forexample,thefollowingqueriescannowusetwoindependentindexeson value1and value2(thelatterqueryrequiresthatthe
indexesareskiplistindexesduetousageofthe <and >comparisonoperators):
FORdocINcollectionFILTERdoc.value1==42||doc.value2==23RETURNdoc
FORdocINcollectionFILTERdoc.value1<42||doc.value2>23RETURNdoc
Thenewoptimizerrule"sort-in-values"cannowpre-sorttheright-handsideoperandof INand NOTINoperatorssotheoperation
canuseabinarysearchwithlogarithmiccomplexityinsteadofalinearsearch.Therulewillbeappliedwhentheright-handsideoperand
ofan INor NOTINoperatorinafilterconditionisavariablethatisdefinedinadifferentloop/scopethantheoperatoritself.
Additionally,thefilterconditionmustconsistofsolelythe INor NOTINoperationinordertoavoidanyside-effects.
Therulewillkickinforaqueriessuchasthefollowing:
LETvalues=/*someruntimeexpressionhere*/
FORdocINcollection
FILTERdoc.valueINvalues
RETURNdoc
Itwillnotbeappliedforthefollowigqueries,becausetheright-handsideoperandofthe INiseithernotavariable,orbecausethe
FILTERconditionmayhavesideeffects:
502
WhatsNewin2.8
FORdocINcollection
FILTERdoc.valueIN/*someruntimeexpressionhere*/
RETURNdoc
LETvalues=/*someruntimeexpressionhere*/
FORdocINcollection
FILTERFUNCTION(doc.values)==23&&doc.valueINvalues
RETURNdoc
AQLfunctionsadded
ThefollowingAQLfunctionshavebeenaddedin2.8:
POW(base,exponent):returnsthebasetotheexponentexp
UNSET_RECURSIVE(document,attributename,...):recursivelyremovestheattributesattributename(canbeoneormany)from
documentanditssub-documents.Allotherattributeswillbepreserved.M ultipleattributenamescanbespecifiedbyeitherpassing
multipleindividualstringargumentnames,orbypassinganarrayofattributenames:
UNSET_RECURSIVE(doc,'_id','_key','foo','bar')
UNSET_RECURSIVE(doc,['_id','_key','foo','bar'])
IS_DATESTRING(value):returnstrueifvalueisastringthatcanbeusedinadatefunction.Thisincludespartialdatessuchas2015
or2015-10andstringscontaininginvaliddatessuchas2015-02-31.Thefunctionwillreturnfalseforallnon-stringvalues,evenif
someofthemmaybeusableindatefunctions.
Miscellaneousimprovements
theArangoShellnowprovidestheconveniencefunction db._explain(query)forretrievingahuman-readableexplanationofAQL
queries.Thisfunctionisashorthandfor require("org/arangodb/aql/explainer").explain(query).
theAQLqueryoptimizernowautomaticallyconverts LENGTH(collection-name)toanoptimizedexpressionthatreturnsthe
numberofdocumentsinacollection.PreviousversionsofArangoDBreturnedawarningwhenusingthisexpressionandalso
enumeratedalldocumentsinthecollection,whichwasinefficient.
improvedperformanceofskippingovermanydocumentsinanAQLquerywhennoindexesandnofiltersareused,e.g.
FORdocINcollection
LIMIT1000000,10
RETURNdoc
addedclusterexecutionsiteinfoinexecutionplanexplainoutputforAQLqueries
for30+AQLfunctionsthereisnowanadditionalimplementationinC++thatremovestheneedforinternaldataconversionwhen
thefunctioniscalled
theAQLeditorinthewebinterfacenowsupportsusingbindparameters
Deadlockdetection
ArangoDB2.8nowhasanautomaticdeadlockdetectionfortransactions.
Adeadlockisasituationinwhichtwoormoreconcurrentoperations(usertransactionsorAQLqueries)trytoaccessthesameresources
(collections,documents)andneedtowaitfortheotherstofinish,butnoneofthemcanmakeanyprogress.
Incaseofsuchadeadlock,therewouldbenoprogressforanyoftheinvolvedtransactions,andnoneoftheinvolvedtransactionscould
evercomplete.Thisiscompletelyundesirable,sothenewautomaticdeadlockdetectionmechanisminArangoDBwillautomaticallykick
inandabortoneofthetransactionsinvolvedinsuchadeadlock.Abortingmeansthatallchangesdonebythetransactionwillberolled
backanderror29( deadlockdetected)willbethrown.
503
WhatsNewin2.8
Clientcode(AQLqueries,usertransactions)thataccessesmorethanonecollectionshouldbeawareofthepotentialofdeadlocksand
shouldhandletheerror29( deadlockdetected)properly,eitherbypassingtheexceptiontothecallerorretryingtheoperation.
Replication
Thefollowingimprovementsforreplicationhavebeenmadein2.8(note:mostofthemhavebeenbackportedtoArangoDB2.7aswell):
added autoResyncconfigurationparameterforcontinuousreplication.
Whensetto true,areplicationslavewillautomaticallytriggerafulldatare-synchronizationwiththemasterwhenthemaster
cannotprovidethelogdatatheslavehadaskedfor.Notethat autoResyncwillonlyworkwhentheoption requireFromPresentis
alsosetto trueforthecontinuousreplication,orwhenthecontinuoussyncerisstartedanddetectsthatnostarttickispresent.
Automaticre-synchronizationmaytransferalotofdatafromthemastertotheslaveandmaybeexpensive.Itisthereforeturnedoff
bydefault.Whenturnedoff,theslavewillneverperformanautomaticre-synchronizationwiththemaster.
added idleMinWaitTimeand idleMaxWaitTimeconfigurationparametersforcontinuousreplication.
Theseparameterscanbeusedtocontroltheminimumandmaximumwaittimetheslavewill(intentionally)idleandnotpollfor
masterlogchangesincasethemasterhadsentthefulllogsalready.The idleMaxWaitTimevaluewillonlybeusedwhen
adapativePollingissetto true.When adaptivePollingisdisabled,only idleMinWaitTimewillbeusedasaconstanttime
spaninwhichtheslavewillnotpollthemasterforfurtherchanges.Thedefaultvaluesare0.5secondsfor idleMinWaitTimeand2.5
secondsfor idleMaxWaitTime,whichcorrespondtothehard-codedvaluesusedinpreviousversionsofArangoDB.
added initialSyncMaxWaitTimeconfigurationparameterforinitialandcontinuousreplication
Thisoptioncontrolsthemaximumwaittime(inseconds)thattheinitialsynchronizationwillwaitforaresponsefromthemaster
whenfetchinginitialcollectiondata.Ifnoresponseisreceivedwithinthistimeperiod,theinitialsynchronizationwillgiveupand
fail.ThisoptionisalsorelevantforcontinuousreplicationincaseautoResyncissettotrue,asthenthecontinuousreplicationmay
triggerafulldatare-synchronizationincasethemastercannotthelogdatatheslavehadaskedfor.
HTTPrequestssentfromtheslavetothemasterduringinitialsynchronizationwillnowberetriediftheyfailwithconnection
problems.
theinitialsynchronizationnowlogsitsprogresssoitcanbequeriedusingtheregularreplicationstatuscheckAPIs.
added asyncattributefor syncand syncCollectionoperationscalledfromtheArangoShell.Setthingthisattributeto truewill
makethesynchronizationjobontheservergointothebackground,sothattheshelldoesnotblock.Thestatusofthestarted
asynchronoussynchronizationjobcanbequeriedfromtheArangoShelllikethis:
/*startsinitialsynchronization*/
varreplication=require("org/arangodb/replication");
varid=replication.sync({
endpoint:"tcp://master.domain.org:8529",
username:"myuser",
password:"mypasswd",
async:true
});
/*nowquerytheidofthereturnedasyncjobandprintthestatus*/
print(replication.getSyncResult(id));
Theresultof getSyncResult()willbe falsewhiletheserver-sidejobhasnotcompleted,anddifferentto falseifithas
completed.Whenithascompleted,alljobresultdetailswillbereturnedbythecallto getSyncResult().
thewebadmininterfacedashboardnowshowsaserver'sreplicationstatusatthebottomofthepage
WebAdminInterface
Thefollowingimprovementshavebeenmadeforthewebadmininterface:
theAQLeditornowhassupportforbindparameters.Thebindparametervaluescanbeeditedinthewebinterfaceandsavedwitha
queryforfutureuse.
504
WhatsNewin2.8
theAQLeditornowallowscancelingrunningqueries.Thiscanbeusedtocancellong-runningquerieswithoutswitchingtothequery
managementsection.
thedashboardnowprovidesinformationabouttheserver'sreplicationstatusatthebottomofthepage.Thiscanbeusedtotrack
eitherthestatusofaone-timesynchronizationorthecontinuousreplication.
thecompactionstatusandsomestatusinternalsaboutcollectionsarenowdisplayedinthedetailviewforacollectionintheweb
interface.Thesedatacanbeusedfordebuggingcompactionissues.
unloadingacollectionviathewebinterfacewillnowtriggergarbagecollectioninallv8contextsandforceaWALflush.This
increasesthechancesofperfomingtheunloadfaster.
thestatusterminologyforcollectionsforwhichanunloadrequesthasbeenissuedviathewebinterfacewaschangedfrom inthe
processofbeingunloadedto willbeunloaded.Thisismoreaccurateastheactualunloadmaybepostponeduntillaterifthere
arestillreferencespointingtodatainthecollection.
Foxximprovements
themoduleresolutionusedby requirenowbehavesmorelikeinnode.js
the org/arangodb/requestmodulenowreturnsresponsebodiesforerrorresponsesbydefault.Theoldbehaviorofnotreturning
bodiesforerrorresponsescanbere-enabledbyexplicitlysettingtheoption returnBodyOnErrorto false
Miscellaneouschanges
Thestartupoption --server.hide-product-headercanbeusedtomaketheservernotsendtheHTTPresponseheader "Server:
ArangoDB"initsHTTPresponses.ThiscanbeusedtoconcealtheservermakefromHTTPclients.Bydefault,theoptionisturnedoff
sotheheaderisstillsentasusual.
arangodumpandarangorestorenowhavebettererrorreporting.Additionally,arangodumpwillnowfailbydefaultwhentryingtodump
edgesthatrefertoalreadydroppedcollections.Thiscanbecircumventedbyspecifyingtheoption --forcetruewheninvoking
arangodump.
arangoimpnowprovidesanoption --create-collection-typetospecifythetypeofthecollectiontobecreatedwhen --createcollectionissetto true.Previously --create-collectionalwayscreateddocumentcollectionsandthecreationofedgecollections
wasnotpossible.
505
Incompatiblechangesin2.8
IncompatiblechangesinArangoDB2.8
ItisrecommendedtocheckthefollowinglistofincompatiblechangesbeforeupgradingtoArangoDB2.8,andadjustanyclientprograms
ifnecessary.
AQL
Keywordsadded
ThefollowingAQLkeywordswereaddedinArangoDB2.8:
GRAPH
OUTBOUND
INBOUND
ANY
ALL
NONE
AGGREGATE
Usageofthesekeywordsforcollectionnames,variablenamesorattributenamesinAQLquerieswillnotbepossiblewithoutquoting.
Forexample,thefollowingAQLquerywillstillworkasitusesaquotedcollectionnameandaquotedattributename:
FORdocIN`OUTBOUND`
RETURNdoc.`any`
Changedbehavior
TheAQLfunctions NEARand WITHINnowhavestrictervalidationsfortheirinputparameters limit, radiusand distance.They
maynowthrowexceptionswheninvalidparametersarepassedthatmayhavenotledtoexceptionsinpreviousversions.
Additionally,theexpansion( [*])operatorinAQLhaschangeditsbehaviorwhenhandlingnon-arrayvalues:
InArangoDB2.8,callingtheexpansionoperatoronanon-arrayvaluewillalwaysreturnanemptyarray.PreviousversionsofArangoDB
expandednon-arrayvaluesbycallingthe TO_ARRAY()functionforthevalue,whichforexamplereturnedanarraywithasinglevaluefor
boolean,numericandstringinputvalues,andanarraywiththeobject'svaluesforanobjectinputvalue.Thisbehaviorwasinconsistent
withhowtheexpansionoperatorworksforthearrayindexesin2.8,sothebehaviorisnowunified:
iftheleft-handsideoperandof [*]isanarray,thearraywillbereturnedasiswhencalling [*]onit
iftheleft-handsideoperandof [*]isnotanarray,anemptyarraywillbereturnedby [*]
AQLqueriesthatrelyontheoldbehaviorcanbechangedbyeithercalling TO_ARRAYexplicitlyorbyusingthe [*]atthecorrect
position.
Thefollowingexamplequerywillchangeitsresultin2.8comparedto2.7:
LETvalues="foo"RETURNvalues[*]
In2.7thequeryhasreturnedthearray ["foo"],butin2.8itwillreturnanemptyarray [].Tomakeitreturnthearray ["foo"
]again,anexplicit TO_ARRAYfunctioncallisneededin2.8(whichinthiscaseallowstheremovalofthe [*]operatoraltogether).This
alsoworksin2.7:
LETvalues="foo"RETURNTO_ARRAY(values)
Anotherexample:
LETvalues=[{name:"foo"},{name:"bar"}]
RETURNvalues[*].name[*]
506
Incompatiblechangesin2.8
Theabovereturned [["foo"],["bar"]]in2.7.In2.8itwillreturn[[],[]] ,becausethevalueofname`isnotanarray.
Tochangetheresultstothe2.7style,thequerycanbechangedto
LETvalues=[{name:"foo"},{name:"bar"}]
RETURNvalues[*RETURNTO_ARRAY(CURRENT.name)]
Theabovealsoworksin2.7.Thefollowingtypesofquerieswon'tchange:
LETvalues=[1,2,3]RETURNvalues[*]
LETvalues=[{name:"foo"},{name:"bar"}]RETURNvalues[*].name
LETvalues=[{names:["foo","bar"]},{names:["baz"]}]RETURNvalues[*].names[*]
LETvalues=[{names:["foo","bar"]},{names:["baz"]}]RETURNvalues[*].names[**]
Deadlockhandling
Clientapplicationsshouldbepreparedtohandleerror29( deadlockdetected)thatArangoDBmaynowthrowwhenitdetectsa
deadlockacrossmultipletransactions.Whenaclientapplicationreceiveserror29,itshouldretrytheoperationthatfailed.
TheerrorcanonlyoccurforAQLqueriesorusertransactionsthatinvolvemorethanasinglecollection.
Optimizer
TheAQLexecutionnodetype IndexRangeNodewasreplacedwithanewmorecapableexecutionnodetype IndexNode.Thatmeansin
executionplanexplainoutputtherewillbenomore IndexRangeNodesbutonly IndexNode.Thisaffectsexplainoutputthatcanbe
retrievedvia require("org/arangodb/aql/explainer").explain(query), db._explain(query),andtheHTTPqueryexplainAPI.
TheoptimizerrulethatmakesAQLqueriesactuallyuseindexeswasalsorenamedfrom use-index-rangeto use-indexes.Againthis
affectsexplainoutputthatcanberetrievedvia require("org/arangodb/aql/explainer").explain(query), db._explain(query),andthe
HTTPqueryexplainAPI.
Thequeryoptimizerrule remove-collect-intowasrenamedto remove-collect-variables.Thisaffectsexplainoutputthatcanbe
retrievedvia require("org/arangodb/aql/explainer").explain(query), db._explain(query),andtheHTTPqueryexplainAPI.
HTTPAPI
Whenaserver-sideoperationgotcanceledduetoanexplicitclientcancelrequestviaHTTP DELETE/_api/job,previousversionsof
ArangoDBreturnedanHTTPstatuscodeof408(requesttimeout)fortheresponseofthecanceledoperation.
TheHTTPreturncode408hascausedproblemswithsomeclientapplications.Somebrowsers(e.g.Chrome)handleda408responseby
resendingtheoriginalrequest,whichistheoppositeofwhatisdesiredwhenajobshouldbecanceled.
ThereforeArangoDBwillreturnHTTPstatuscode410(gone)forcanceledoperationsfromversion2.8on.
Foxx
ModelandRepository
DuetocompatibilityissuestheM odelandRepositorytypesarenolongerimplementedasES2015classes.
Thepre-2.7"extend"stylesubclassingissupportedagainandwillnotemitanydeprecationwarnings.
varFoxx=require('org/arangodb/foxx');
varMyModel=Foxx.Model.extend({
//...
schema:{/*...*/}
});
Moduleresolution
507
Incompatiblechangesin2.8
ThebehavioroftheJavaScriptmoduleresolutionusedbythe requirefunctionhasbeenmodifiedtoimprovecompatibilitywith
moduleswrittenforNode.js.
Specifically
absolutepaths(e.g. /some/absolute/path)arenowalwaysinterpretedasabsolutefilesystempaths,relativetothefilesystemroot
globalnames(e.g. global/name)arenowfirstintepretedasreferencestomodulesresidinginarelevant node_modulesfolder,a
built-inmoduleoramatchingdocumentintheinternal _modulescollection,andonlyresolvedtolocalfilepathsifnoothermatchis
found
Previouslythetwoformatsweretreatedinterchangeablyandwouldberesolvedtolocalfilepathsfirst,leadingtoproblemswhenlocal
filesusedthesamenamesasothermodules(e.g.alocalfile chai.jswouldcauseproblemswhentryingtoloadthe chaimodule
installedin node_modules).
Formoreinformationseetheblogannouncementofthischangeandtheupgradeguide.
Module org/arangodb/request
Themodulenowalwaysreturnsresponsebodies,evenforerrorresponses.Inversionspriorto2.8themodulewouldsilentlydrop
responsebodiesiftheresponseheaderindicatedanerror.
Theoldbehaviorofnotreturningbodiesforerrorresponsescanberestoredbyexplicitlysettingtheoption returnBodyOnErrorto
false:
letresponse=request({
//...
returnBodyOnError:false
});
Garbagecollection
TheV8garbagecollectionstrategywasslightlyadjustedsothatiteventuallyhappensinallV8contextsthatholdV8externalobjects
(referencestoArangoDBdocumentsandcollections).Thisenablesabettercleanupoftheseresourcesandpreventsotherprocessessuch
ascompactionbeingstalledwhilewaitingfortheseresourcestobereleased.
InthiscontextthedefaultvaluefortheJavaScriptgarbagecollectionfrequency( --javascript.gc-frequency)wasalsoincreasedfrom10
secondsto15seconds,aslessinternaloperationsinArangoDBarecarriedoutinJavaScript.
Clienttools
arangodumpwillnowfailbydefaultwhentryingtodumpedgesthatrefertoalreadydroppedcollections.Thiscanbecircumventedby
specifyingtheoption --forcetruewheninvokingarangodump
508
WhatsNewin2.7
FeaturesandImprovements
ThefollowinglistshowsindetailwhichfeatureshavebeenaddedorimprovedinArangoDB2.7.ArangoDB2.7alsocontainsseveral
bugfixesthatarenotlistedhere.Foralistofbugfixes,pleaseconsulttheCHANGELOG.
Performanceimprovements
Indexbuckets
Theprimaryindexesandhashindexesofcollectionscannowbesplitintomultipleindexbuckets.Thisoptionwasavailableforedge
indexesonlyinArangoDB2.6.
Abucketcanbeconsideredacontainerforaspecificrangeofindexvalues.Forprimary,hashandedgeindexes,determiningthe
responsiblebucketforanindexvalueisdonebyhashingtheactualindexvalueandapplyingasimplearithmeticoperationonthehash.
Becauseanindexvaluewillbepresentinatmostonebucketandbucketsareindependent,usingmultiplebucketsprovidesthefollowing
benefits:
initiallybuildingthein-memoryindexdatacanbeparallelizedevenforasingleindex,withonethreadperbucket(orwiththreads
beingresponsibleformorethanonebucketatatime).Thiscanhelpreducingtheloadingtimeforcollections.
resizinganindexwhenitisabouttorunoutofreservespaceisperformedperbucket.Aseachbucketonlycontainsafractionofthe
entireindex,resizingandrehashingabucketismuchfasterandlessintrusivethanresizingandrehashingtheentireindex.
Whencreatingnewcollections,thedefaultnumberofindexbucketsis 8sinceArangoDB2.7.Inpreviousversions,thedefaultvalue
was 1.Thenumberofbucketscanalsobeadjustedforexistingcollectionssotheycanbenefitfromtheoptimizations.Thenumberof
indexbucketscanbesetforacollectionatanytimebyusingacollection's propertiesfunction:
db.collection.properties({indexBuckets:16});
Thenumberofindexbucketsmustbeapowerof2.
Pleasenotethatforbuildingtheindexdataformultiplebucketsinparallelitisrequiredthatacollectioncontainsasignificantamountof
documentsbecauseforalownumberofdocumentstheoverheadofparallelizationwilloutweighitsbenefits.Thecurrentthresholdvalue
is256kdocuments,butthisvaluemaychangeinfutureversionsofArangoDB.Additionally,theconfigurationoption --database.indexthreadswilldeterminehowmanyparallelthreadsmaybeusedforbuildingtheindexdata.
Fasterupdateandremoveoperationsinnon-uniquehashindexes
TheuniquehashindexesinArangoDBprovidedanamortizedO(1)lookup,insert,updateandremoveperformance.Non-uniquehash
indexesprovidedamortizedO(1)insertperformance,buthadworseperformanceforupdateandremoveoperationsfornon-unique
values.Fordocumentswiththesameindexvalue,theymaintainedalistofcollisions.Whenadocumentwasupdatedorremoved,that
exactdocumenthadtobefoundinthecollisionslistfortheindexvalue.WhilegettingtothestartofthecollisionslistwasO(1),scanning
thelisthadO(n)performanceintheworstcase(withnbeingthenumberofdocumentswiththesameindexvalue).Overall,thismade
updateandremoveoperationsinnon-uniquehashindexesslowiftheindexcontainedmanyduplicatevalues.
ThishasbeenchangedinArangoDB2.7sothatnon-uniquehashindexesnowalsoprovideupdateandremoveoperationswithan
amortizedcomplexityofO(1),eveniftherearemanyduplicates.
Resizingnon-uniquehashindexesnowalsodoesn'trequirelookingintothedocumentdata(whichmayinvolveadiskaccess)becausethe
indexmaintainssomeinternalcachevalueperdocument.Whenresizingandrehashingtheindex(oranindexbucket),theindexwillfirst
compareonlythecachevaluesbeforepeekingintotheactualdocuments.Thischangecanalsoleadtoreducedindexresizingtimes.
Throughputenhancements
TheArangoDB-internalimplementationsfordispatchingrequests,keepingstatisticsandassigningV8contextstothreadshavebeen
improvedinordertouselesslocks.Thesechangesallowhigherconcurrencyandthroughputinthesecomponents,whichcanalsomake
theserverhandlemorerequestsinagivenperiodoftime.
509
WhatsNewin2.7
Whatgainscanbeexpecteddependsonwhichoperationsareexecuted,buttherearereal-worldcasesinwhichthroughputincreasedby
between25%and70%whencomparedto2.6.
Madvisehints
TheLinuxvariantforArangoDBprovidestheOSwithmadvisehintsaboutindexmemoryanddatafilememory.Thesehintscanspeed
upthingswhenmemoryistight,inparticularatcollectionloadtimebutalsoforrandomaccesseslater.Thereisnoformalguaranteethat
theOSactuallyusesthemadvisehintsprovidedbyArangoDB,butactualmeasurementshaveshownimprovementsforloadingbigger
collections.
AQLimprovements
Additionaldatefunctions
ArangoDB2.7providesseveralextraAQLfunctionsfordateandtimecalculationandmanipulation.Thesefunctionswerecontributedby
GitHubusers@CoDEmanXand@friday.Abigthanksfortheirwork!
Thefollowingextradatefunctionsareavailablefrom2.7on:
DATE_DAYOFYEAR(date):Returnsthedayofyearnumberofdate.Thereturnvaluesrangefrom1to365,or366inaleapyear
respectively.
DATE_ISOWEEK(date):ReturnstheISOweekdateofdate.Thereturnvaluesrangefrom1to53.M ondayisconsideredthefirstday
oftheweek.Therearenofractionalweeks,thusthelastdaysinDecembermaybelongtothefirstweekofthenextyear,andthe
firstdaysinJanuarymaybepartofthepreviousyear'slastweek.
DATE_LEAPYEAR(date):Returnswhethertheyearofdateisaleapyear.
DATE_QUARTER(date):Returnsthequarterofthegivendate(1-based):
1:January,February,M arch
2:April,M ay,June
3:July,August,September
4:October,Novemb
Download PDF
Similar pages