AryaWu/sqlite
0
1<h1 align="center">SQLite Source Repository</h1>2 3This repository contains the complete source code for the4[SQLite database engine](https://sqlite.org/) going back5to 2000-05-29. The tree includes many tests and some6documentation, though additional tests and most documentation7are managed separately.8 9See the [on-line documentation](https://sqlite.org/) for more information10about what SQLite is and how it works from a user's perspective. This11README file is about the source code that goes into building SQLite,12not about how SQLite is used.13 14## Version Control15 16SQLite sources are managed using17[Fossil](https://fossil-scm.org/), a distributed version control system18that was specifically designed and written to support SQLite development.19The [Fossil repository](https://sqlite.org/src/timeline) contains the urtext.20 21If you are reading this on GitHub or some other Git repository or service,22then you are looking at a mirror. The names of check-ins and23other artifacts in a Git mirror are different from the official24names for those objects. The official names for check-ins are25found in a footer on the check-in comment for authorized mirrors.26The official check-in name can also be seen in the `manifest.uuid` file27in the root of the tree. Always use the official name, not the28Git-name, when communicating about an SQLite check-in.29 30If you pulled your SQLite source code from a secondary source and want to31verify its integrity, there are hints on how to do that in the32[Verifying Code Authenticity](#vauth) section below.33 34## Contacting The SQLite Developers35 36The preferred way to ask questions or make comments about SQLite or to37report bugs against SQLite is to visit the 38[SQLite Forum](https://sqlite.org/forum) at <https://sqlite.org/forum/>.39Anonymous postings are permitted.40 41If you think you have found a bug that has security implications and42you do not want to report it on the public forum, you can send a private43email to drh at sqlite dot org.44 45## Public Domain46 47The SQLite source code is in the public domain. See48<https://sqlite.org/copyright.html> for details. 49 50Because SQLite is in the public domain, we do not normally accept pull51requests, because if we did take a pull request, the changes in that52pull request might carry a copyright and the SQLite source code would53then no longer be fully in the public domain.54 55## Obtaining The SQLite Source Code56 57Source code tarballs or ZIP archives are available at:58 59 * [Latest trunk check-in](https://sqlite.org/src/rchvdwnld/trunk).60 61 * [Latest release](https://sqlite.org/src/rchvdwnld/release)62 63 * For other check-ins, browse the64 [project timeline](https://sqlite.org/src/timeline?y=ci) and65 click on the check-in hash of the check-in you want to download.66 On the resulting "info" page, click one of the options to the67 right of the "**Downloads:**" label in the "**Overview**" section68 near the top.69 70To access sources directly using [Fossil](https://fossil-scm.org/home),71first install Fossil version 2.0 or later.72Source tarballs and precompiled binaries for Fossil are available at73<https://fossil-scm.org/home/uv/download.html>. Fossil is74a stand-alone program. To install, simply download or build the single75executable file and put that file someplace on your $PATH or %PATH%.76Then run commands like this:77 78 mkdir -p ~/sqlite79 cd ~/sqlite80 fossil open https://sqlite.org/src81 82The initial "fossil open" command will take two or three minutes. Afterwards,83you can do fast, bandwidth-efficient updates to the whatever versions84of SQLite you like. Some examples:85 86 fossil update trunk ;# latest trunk check-in87 fossil update release ;# latest official release88 fossil update trunk:2024-01-01 ;# First trunk check-in after 2024-01-0189 fossil update version-3.39.0 ;# Version 3.39.090 91Or type "fossil ui" to get a web-based user interface.92 93## Compiling for Unix-like systems94 95First create a directory in which to place96the build products. It is recommended, but not required, that the97build directory be separate from the source directory. Cd into the98build directory and then from the build directory run the configure99script found at the root of the source tree. Then run "make".100 101For example:102 103 apt install gcc make tcl-dev ;# Install the necessary build tools104 tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite"105 mkdir bld ;# Build happens in a sibling directory106 cd bld ;# Change to the build directory107 ../sqlite/configure ;# Run the configure script108 make sqlite3 ;# The "sqlite3" command-line tool109 make sqlite3.c ;# The "amalgamation" source file110 make sqldiff ;# The "sqldiff" command-line tool111 #### Targets below require tcl-dev ####112 make tclextension-install ;# Install the SQLite TCL extension113 make devtest ;# Run development tests114 make releasetest ;# Run full release tests115 make sqlite3_analyzer ;# Builds the "sqlite3_analyzer" tool116 117See the makefile for additional targets. For debugging builds, the118core developers typically run "configure" with options like this:119 120 ../sqlite/configure --all --debug CFLAGS='-O0 -g'121 122For release builds, the core developers usually do:123 124 ../sqlite/configure --all125 126Core deliverables (sqlite3.c, sqlite3) can be built without a TCL, but127many makefile targets require a "tclsh" TCL interpreter version 8.6128or later. The "tclextension-install" target and the test targets that follow129all require TCL development libraries too. ("apt install tcl-dev"). It is130helpful, but is not required, to install the SQLite TCL extension (the131"tclextension-install" target) prior to running tests. The "releasetest"132target has additional requirements, such as "valgrind".133 134On "make" command-lines, one can add "OPTIONS=..." to specify additional135compile-time options over and above those set by ./configure. For example,136to compile with the SQLITE_OMIT_DEPRECATED compile-time option, one could say:137 138 ./configure --all139 make OPTIONS=-DSQLITE_OMIT_DEPRECATED sqlite3140 141The configure script uses [autosetup](https://msteveb.github.io/autosetup/).142If the configure script does not work out for you, there is a generic143makefile named "Makefile.linux-gcc" in the top directory of the source tree144that you can copy and edit to suit your needs. Comments on the generic145makefile show what changes are needed.146 147## Compiling for Windows Using MSVC148 149On Windows, everything can be compiled with MSVC.150You will also need a working installation of TCL if you want to run tests,151though TCL is not required if you just want to build SQLite itself.152See the [compile-for-windows.md](doc/compile-for-windows.md) document for153additional information about how to install MSVC and TCL and configure your154build environment.155 156If you want to run tests, you need to let SQLite know the location of your157TCL library, using a command like this:158 159 set TCLDIR=c:\Tcl160 161SQLite itself does not contain any TCL code, but it does use TCL to run162tests. You may need to install TCL development libraries in order to163successfully complete some makefile targets. It is helpful, but is not164required, to install the SQLite TCL extension (the "tclextension-install"165target) prior to running tests.166 167The source tree contains a "make.bat" file that allows the same "make"168commands of Unix to work on Windows. In the following, you can substitute169"nmake /f Makefile.msc" in place of "make", if you prefer to avoid this BAT170file:171 172 make sqlite3.exe173 make sqlite3.c174 make sqldiff.exe175 #### Targets below require TCL development libraries ####176 make tclextension-install177 make devtest178 make releasetest179 make sqlite3_analyzer.exe180 181There are many other makefile targets. See comments in Makefile.msc for182details.183 184As with the unix Makefile, the OPTIONS=... argument can be passed on the nmake185command-line to enable new compile-time options. For example:186 187 make OPTIONS=-DSQLITE_OMIT_DEPRECATED sqlite3.exe188 189## Source Tree Map190 191 * **src/** - This directory contains the primary source code for the192 SQLite core. For historical reasons, C-code used for testing is193 also found here. Source files intended for testing begin with "`test`".194 The `tclsqlite3.c` and `tclsqlite3.h` files are the TCL interface195 for SQLite and are also not part of the core.196 197 * **test/** - This directory and its subdirectories contains code used198 for testing. Files that end in "`.test`" are TCL scripts that run199 tests using an augmented TCL interpreter named "testfixture". Use200 a command like "`make testfixture`" to build that201 augmented TCL interpreter, then run individual tests using commands like202 "`testfixture test/main.test`". This test/ subdirectory also contains203 additional C code modules and scripts for other kinds of testing.204 205 * **tool/** - This directory contains programs and scripts used to206 build some of the machine-generated code that goes into the SQLite207 core, as well as to build and run tests and perform diagnostics.208 The source code to [the Lemon parser generator](./doc/lemon.html) is209 found here. There are also TCL scripts used to build and/or transform210 source code files. For example, the tool/mksqlite3h.tcl script reads211 the src/sqlite.h.in file and uses it as a template to construct212 the deliverable "sqlite3.h" file that defines the SQLite interface.213 214 * **ext/** - Various extensions to SQLite are found under this215 directory. For example, the FTS5 subsystem is in "ext/fts5/".216 Some of these extensions (ex: FTS3/4, FTS5, RTREE) might get built217 into the SQLite amalgamation, but not all of them. The218 "ext/misc/" subdirectory contains an assortment of one-file extensions,219 many of which are omitted from the SQLite core, but which are included220 in the [SQLite CLI](https://sqlite.org/cli.html).221 222 * **doc/** - Some documentation files about SQLite internals are found223 here. Note, however, that the primary documentation designed for224 application developers and users of SQLite is in a completely separate225 repository. Note also that the primary API documentation is derived226 from specially constructed comments in the src/sqlite.h.in file.227 228### Generated Source Code Files229 230Several of the C-language source files used by SQLite are generated from231other sources rather than being typed in manually by a programmer. This232section will summarize those automatically-generated files. To create all233of the automatically-generated files, simply run "make target_source".234The "target_source" make target will create a subdirectory "tsrc/" and235fill it with all the source files needed to build SQLite, both236manually-edited files and automatically-generated files.237 238The SQLite interface is defined by the **sqlite3.h** header file, which is239generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The240[Tcl script](https://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion.241The manifest.uuid file contains the SHA3 hash of the particular check-in242and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file243contains the current SQLite version number. The sqlite3.h header is really244just a copy of src/sqlite.h.in with the source-id and version number inserted245at just the right spots. Note that comment text in the sqlite3.h file is246used to generate much of the SQLite API documentation. The Tcl scripts247used to generate that documentation are in a separate source repository.248 249The SQL language parser is **parse.c** which is generated from a grammar in250the src/parse.y file. The conversion of "parse.y" into "parse.c" is done251by the [lemon](./doc/lemon.html) LALR(1) parser generator. The source code252for lemon is at tool/lemon.c. Lemon uses the tool/lempar.c file as a253template for generating its parser.254Lemon also generates the **parse.h** header file, at the same time it255generates parse.c.256 257The **opcodes.h** header file contains macros that define the numbers258corresponding to opcodes in the "VDBE" virtual machine. The opcodes.h259file is generated by scanning the src/vdbe.c source file. The260Tcl script at ./mkopcodeh.tcl does this scan and generates opcodes.h.261A second Tcl script, ./mkopcodec.tcl, then scans opcodes.h to generate262the **opcodes.c** source file, which contains a reverse mapping from263opcode-number to opcode-name that is used for EXPLAIN output.264 265The **keywordhash.h** header file contains the definition of a hash table266that maps SQL language keywords (ex: "CREATE", "SELECT", "INDEX", etc.) into267the numeric codes used by the parse.c parser. The keywordhash.h file is268generated by a C-language program at tool mkkeywordhash.c.269 270The **pragma.h** header file contains various definitions used to parse271and implement the PRAGMA statements. The header is generated by a272script **tool/mkpragmatab.tcl**. If you want to add a new PRAGMA, edit273the **tool/mkpragmatab.tcl** file to insert the information needed by the274parser for your new PRAGMA, then run the script to regenerate the275**pragma.h** header file.276 277### The Amalgamation278 279All of the individual C source code and header files (both manually-edited280and automatically-generated) can be combined into a single big source file281**sqlite3.c** called "the amalgamation". The amalgamation is the recommended282way of using SQLite in a larger application. Combining all individual283source code files into a single big source code file allows the C compiler284to perform more cross-procedure analysis and generate better code. SQLite285runs about 5% faster when compiled from the amalgamation versus when compiled286from individual source files.287 288The amalgamation is generated from the tool/mksqlite3c.tcl Tcl script.289First, all of the individual source files must be gathered into the tsrc/290subdirectory (using the equivalent of "make target_source") then the291tool/mksqlite3c.tcl script is run to copy them all together in just the292right order while resolving internal "#include" references.293 294The amalgamation source file is more than 200K lines long. Some symbolic295debuggers (most notably MSVC) are unable to deal with files longer than 64K296lines. To work around this, a separate Tcl script, tool/split-sqlite3c.tcl,297can be run on the amalgamation to break it up into a single small C file298called **sqlite3-all.c** that does #include on about seven other files299named **sqlite3-1.c**, **sqlite3-2.c**, ..., **sqlite3-7.c**. In this way,300all of the source code is contained within a single translation unit so301that the compiler can do extra cross-procedure optimization, but no302individual source file exceeds 32K lines in length.303 304## How It All Fits Together305 306SQLite is modular in design.307See the [architectural description](https://sqlite.org/arch.html)308for details. Other documents that are useful in309helping to understand how SQLite works include the310[file format](https://sqlite.org/fileformat2.html) description,311the [virtual machine](https://sqlite.org/opcode.html) that runs312prepared statements, the description of313[how transactions work](https://sqlite.org/atomiccommit.html), and314the [overview of the query planner](https://sqlite.org/optoverview.html).315 316Decades of effort have gone into optimizing SQLite, both317for small size and high performance. And optimizations tend to result in318complex code. So there is a lot of complexity in the current SQLite319implementation. It will not be the easiest library in the world to hack.320 321### Key source code files322 323 * **sqlite.h.in** - This file defines the public interface to the SQLite324 library. Readers will need to be familiar with this interface before325 trying to understand how the library works internally. This file is326 really a template that is transformed into the "sqlite3.h" deliverable327 using a script invoked by the makefile.328 329 * **sqliteInt.h** - this header file defines many of the data objects330 used internally by SQLite. In addition to "sqliteInt.h", some331 subsystems inside of sQLite have their own header files. These internal332 interfaces are not for use by applications. They can and do change333 from one release of SQLite to the next.334 335 * **parse.y** - This file describes the LALR(1) grammar that SQLite uses336 to parse SQL statements, and the actions that are taken at each step337 in the parsing process. The file is processed by the338 [Lemon Parser Generator](./doc/lemon.html) to produce the actual C code339 used for parsing.340 341 * **vdbe.c** - This file implements the virtual machine that runs342 prepared statements. There are various helper files whose names343 begin with "vdbe". The VDBE has access to the vdbeInt.h header file344 which defines internal data objects. The rest of SQLite interacts345 with the VDBE through an interface defined by vdbe.h.346 347 * **where.c** - This file (together with its helper files named348 by "where*.c") analyzes the WHERE clause and generates349 virtual machine code to run queries efficiently. This file is350 sometimes called the "query optimizer". It has its own private351 header file, whereInt.h, that defines data objects used internally.352 353 * **btree.c** - This file contains the implementation of the B-Tree354 storage engine used by SQLite. The interface to the rest of the system355 is defined by "btree.h". The "btreeInt.h" header defines objects356 used internally by btree.c and not published to the rest of the system.357 358 * **pager.c** - This file contains the "pager" implementation, the359 module that implements transactions. The "pager.h" header file360 defines the interface between pager.c and the rest of the system.361 362 * **os_unix.c** and **os_win.c** - These two files implement the interface363 between SQLite and the underlying operating system using the run-time364 pluggable VFS interface.365 366 * **shell.c.in** - This file is not part of the core SQLite library. This367 is the file that, when linked against sqlite3.a, generates the368 "sqlite3.exe" command-line shell. The "shell.c.in" file is transformed369 into "shell.c" as part of the build process.370 371 * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It372 is not part of the core SQLite library. But as most of the tests in this373 repository are written in Tcl, the Tcl language bindings are important.374 375 * **test\*.c** - Files in the src/ folder that begin with "test" go into376 building the "testfixture.exe" program. The testfixture.exe program is377 an enhanced Tcl shell. The testfixture.exe program runs scripts in the378 test/ folder to validate the core SQLite code. The testfixture program379 (and some other test programs too) is built and run when you type380 "make test".381 382 * **VERSION**, **manifest**, **manifest.tags**, and **manifest.uuid** -383 These files define the current SQLite version number. The "VERSION" file384 is human generated, but the "manifest", "manifest.tags", and385 "manifest.uuid" files are automatically generated by the386 [Fossil version control system](https://fossil-scm.org/).387 388There are many other source files. Each has a succinct header comment that389describes its purpose and role within the larger system.390 391<a name="vauth"></a>392## Verifying Code Authenticity393 394The `manifest` file at the root directory of the source tree395contains either a SHA3-256 hash or a SHA1 hash396for every source file in the repository.397The name of the version of the entire source tree is just the398SHA3-256 hash of the `manifest` file itself, possibly with the399last line of that file omitted if the last line begins with400"`# Remove this line`".401The `manifest.uuid` file should contain the SHA3-256 hash of the402`manifest` file. If all of the above hash comparisons are correct, then403you can be confident that your source tree is authentic and unadulterated.404Details on the format for the `manifest` files are available405[on the Fossil website](https://fossil-scm.org/home/doc/trunk/www/fileformat.wiki#manifest).406 407The process of checking source code authenticity is automated by the 408makefile:409 410> make verify-source411 412Using the makefile to verify source integrity is good for detecting413accidental changes to the source tree, but malicious changes could be414hidden by also modifying the makefiles.415 416## Contacts417 418The main SQLite website is [https://sqlite.org/](https://sqlite.org/)419with geographically distributed backups at420[https://www2.sqlite.org/](https://www2.sqlite.org) and421[https://www3.sqlite.org/](https://www3.sqlite.org).422 423Contact the SQLite developers through the424[SQLite Forum](https://sqlite.org/forum/). In an emergency, you425can send private email to the lead developer at drh at sqlite dot org.426