AryaWu/sqlite
0
1# 2005 November 302#3# The author disclaims copyright to this source code. In place of4# a legal notice, here is a blessing:5#6# May you do good and not evil.7# May you find forgiveness for yourself and forgive others.8# May you share freely, never taking more than you give.9#10#***********************************************************************11#12# This file contains tests to ensure that the library handles malloc() failures13# correctly. The emphasis of these tests are the _prepare(), _step() and14# _finalize() calls.15#16# $Id: malloc3.test,v 1.24 2008/10/14 15:54:08 drh Exp $17 18set testdir [file dirname $argv0]19source $testdir/tester.tcl20source $testdir/malloc_common.tcl21 22# Only run these tests if memory debugging is turned on.23#24if {!$MEMDEBUG} {25 puts "Skipping malloc3 tests: not compiled with -DSQLITE_MEMDEBUG..."26 finish_test27 return28}29 30# Do not run these tests if F2FS batch writes are supported. In this case,31# it is possible for a single DML statement in an implicit transaction32# to fail with SQLITE_NOMEM, but for the transaction to still end up33# committed to disk. Which confuses the tests in this module.34#35if {[atomic_batch_write test.db]} {36 puts "Skipping malloc3 tests: atomic-batch support"37 finish_test38 return39}40 41 42# Do not run these tests with an in-memory journal.43#44# In the pager layer, if an IO or OOM error occurs during a ROLLBACK, or45# when flushing a page to disk due to cache-stress, the pager enters an46# "error state". The only way out of the error state is to unlock the47# database file and end the transaction, leaving whatever journal and48# database files happen to be on disk in place. The next time the current49# (or any other) connection opens a read transaction, hot-journal rollback50# is performed if necessary.51#52# Of course, this doesn't work with an in-memory journal.53#54if {[permutation]=="inmemory_journal"} {55 finish_test56 return57}58 59#--------------------------------------------------------------------------60# NOTES ON RECOVERING FROM A MALLOC FAILURE61# 62# The tests in this file test the behaviours described in the following63# paragraphs. These tests test the behaviour of the system when malloc() fails64# inside of a call to _prepare(), _step(), _finalize() or _reset(). The65# handling of malloc() failures within ancillary procedures is tested66# elsewhere.67#68# Overview:69#70# Executing a statement is done in three stages (prepare, step and finalize). A71# malloc() failure may occur within any stage. If a memory allocation fails72# during statement preparation, no statement handle is returned. From the users73# point of view the system state is as if _prepare() had never been called.74#75# If the memory allocation fails during the _step() or _finalize() calls, then76# the database may be left in one of two states (after finalize() has been77# called):78#79# * As if the neither _step() nor _finalize() had ever been called on80# the statement handle (i.e. any changes made by the statement are81# rolled back).82# * The current transaction may be rolled back. In this case a hot-journal83# may or may not actually be present in the filesystem.84#85# The caller can tell the difference between these two scenarios by invoking86# _get_autocommit().87#88#89# Handling of sqlite3_reset():90#91# If a malloc() fails while executing an sqlite3_reset() call, this is handled92# in the same way as a failure within _finalize(). The statement handle93# is not deleted and must be passed to _finalize() for resource deallocation.94# Attempting to _step() or _reset() the statement after a failed _reset() will95# always return SQLITE_NOMEM.96#97#98# Other active SQL statements:99#100# The effect of a malloc failure on concurrently executing SQL statements,101# particularly when the statement is executing with READ_UNCOMMITTED set and102# the malloc() failure mandates statement rollback only. Currently, if103# transaction rollback is required, all other vdbe's are aborted.104#105# Non-transient mallocs in btree.c:106# * The Btree structure itself107# * Each BtCursor structure108#109# Mallocs in pager.c:110# readMasterJournal() - Space to read the master journal name111# pager_delmaster() - Space for the entire master journal file112#113# sqlite3pager_open() - The pager structure itself114# sqlite3_pagerget() - Space for a new page115# pager_open_journal() - Pager.aInJournal[] bitmap116# sqlite3pager_write() - For in-memory databases only: history page and117# statement history page.118# pager_stmt_begin() - Pager.aInStmt[] bitmap119#120# None of the above are a huge problem. The most troublesome failures are the121# transient malloc() calls in btree.c, which can occur during the tree-balance122# operation. This means the tree being balanced will be internally inconsistent123# after the malloc() fails. To avoid the corrupt tree being read by a124# READ_UNCOMMITTED query, we have to make sure the transaction or statement125# rollback occurs before sqlite3_step() returns, not during a subsequent126# sqlite3_finalize().127#--------------------------------------------------------------------------128 129#--------------------------------------------------------------------------130# NOTES ON TEST IMPLEMENTATION131#132# The tests in this file are implemented differently from those in other133# files. Instead, tests are specified using three primitives: SQL, PREP and134# TEST. Each primitive has a single argument. Primitives are processed in135# the order they are specified in the file.136#137# A TEST primitive specifies a TCL script as its argument. When a TEST138# directive is encountered the Tcl script is evaluated. Usually, this Tcl139# script contains one or more calls to [do_test].140#141# A PREP primitive specifies an SQL script as its argument. When a PREP142# directive is encountered the SQL is evaluated using database connection143# [db].144#145# The SQL primitives are where the action happens. An SQL primitive must146# contain a single, valid SQL statement as its argument. When an SQL147# primitive is encountered, it is evaluated one or more times to test the148# behaviour of the system when malloc() fails during preparation or149# execution of said statement. The Nth time the statement is executed,150# the Nth malloc is said to fail. The statement is executed until it151# succeeds, i.e. (M+1) times, where M is the number of mallocs() required152# to prepare and execute the statement.153#154# Each time an SQL statement fails, the driver program (see proc [run_test]155# below) figures out if a transaction has been automatically rolled back.156# If not, it executes any TEST block immediately proceeding the SQL157# statement, then reexecutes the SQL statement with the next value of N.158#159# If a transaction has been automatically rolled back, then the driver160# program executes all the SQL specified as part of SQL or PREP primitives161# between the current SQL statement and the most recent "BEGIN". Any 162# TEST block immediately proceeding the SQL statement is evaluated, and163# then the SQL statement reexecuted with the incremented N value.164#165# That make any sense? If not, read the code in [run_test] and it might.166#167# Extra restriction imposed by the implementation:168#169# * If a PREP block starts a transaction, it must finish it.170# * A PREP block may not close a transaction it did not start.171#172#--------------------------------------------------------------------------173 174 175# These procs are used to build up a "program" in global variable176# ::run_test_script. At the end of this file, the proc [run_test] is used177# to execute the program (and all test cases contained therein).178#179set ::run_test_sql_id 0180set ::run_test_script [list]181proc TEST {id t} {lappend ::run_test_script -test [list $id $t]}182proc PREP {p} {lappend ::run_test_script -prep [string trim $p]}183proc DEBUG {s} {lappend ::run_test_script -debug $s}184 185# SQL --186#187# SQL ?-norollback? <sql-text>188#189# Add an 'SQL' primitive to the program (see notes above). If the -norollback190# switch is present, then the statement is not allowed to automatically roll191# back any active transaction if malloc() fails. It must rollback the statement192# transaction only.193#194proc SQL {a1 {a2 ""}} {195 # An SQL primitive parameter is a list of three elements, an id, a boolean196 # value indicating if the statement may cause transaction rollback when197 # malloc() fails, and the sql statement itself.198 set id [incr ::run_test_sql_id]199 if {$a2 == ""} {200 lappend ::run_test_script -sql [list $id true [string trim $a1]]201 } else {202 lappend ::run_test_script -sql [list $id false [string trim $a2]]203 }204}205 206# TEST_AUTOCOMMIT --207# 208# A shorthand test to see if a transaction is active or not. The first209# argument - $id - is the integer number of the test case. The second210# argument is either 1 or 0, the expected value of the auto-commit flag.211#212proc TEST_AUTOCOMMIT {id a} {213 TEST $id "do_test \$testid { sqlite3_get_autocommit \$::DB } {$a}"214}215 216#--------------------------------------------------------------------------217# Start of test program declaration218#219 220 221# Warm body test. A malloc() fails in the middle of a CREATE TABLE statement222# in a single-statement transaction on an empty database. Not too much can go223# wrong here.224#225TEST 1 {226 do_test $testid {227 execsql {SELECT tbl_name FROM sqlite_master;}228 } {}229}230SQL { 231 CREATE TABLE IF NOT EXISTS abc(a, b, c); 232}233TEST 2 {234 do_test $testid.1 {235 execsql {SELECT tbl_name FROM sqlite_master;}236 } {abc}237}238 239# Insert a couple of rows into the table. each insert is in its own240# transaction. test that the table is unpopulated before running the inserts241# (and hence after each failure of the first insert), and that it has been242# populated correctly after the final insert succeeds.243#244TEST 3 {245 do_test $testid.2 {246 execsql {SELECT * FROM abc}247 } {}248}249SQL {INSERT INTO abc VALUES(1, 2, 3);}250SQL {INSERT INTO abc VALUES(4, 5, 6);}251SQL {INSERT INTO abc VALUES(7, 8, 9);}252TEST 4 {253 do_test $testid {254 execsql {SELECT * FROM abc}255 } {1 2 3 4 5 6 7 8 9}256}257 258# Test a CREATE INDEX statement. Because the table 'abc' is so small, the index259# will all fit on a single page, so this doesn't test too much that the CREATE260# TABLE statement didn't test. A few of the transient malloc()s in btree.c261# perhaps.262#263SQL {CREATE INDEX abc_i ON abc(a, b, c);}264TEST 4 {265 do_test $testid {266 execsql {267 SELECT * FROM abc ORDER BY a DESC;268 }269 } {7 8 9 4 5 6 1 2 3}270}271 272# Test a DELETE statement. Also create a trigger and a view, just to make sure273# these statements don't have any obvious malloc() related bugs in them. Note274# that the test above will be executed each time the DELETE fails, so we're275# also testing rollback of a DELETE from a table with an index on it.276#277SQL {DELETE FROM abc WHERE a > 2;}278SQL {CREATE TRIGGER abc_t AFTER INSERT ON abc BEGIN SELECT 'trigger!'; END;}279SQL {CREATE VIEW abc_v AS SELECT * FROM abc;}280TEST 5 {281 do_test $testid {282 execsql {283 SELECT name, tbl_name FROM sqlite_master ORDER BY name;284 SELECT * FROM abc;285 }286 } {abc abc abc_i abc abc_t abc abc_v abc_v 1 2 3}287}288 289set sql {290 BEGIN;DELETE FROM abc;291}292for {set i 1} {$i < 100} {incr i} {293 set a $i294 set b "String value $i"295 set c [string repeat X $i]296 append sql "INSERT INTO abc VALUES ($a, '$b', '$c');"297}298append sql {COMMIT;}299PREP $sql300 301SQL {302 DELETE FROM abc WHERE oid IN (SELECT oid FROM abc ORDER BY random() LIMIT 5);303}304TEST 6 {305 do_test $testid.1 {306 execsql {SELECT count(*) FROM abc}307 } {94}308 do_test $testid.2 {309 execsql {310 SELECT min(311 (oid == a) AND 'String value ' || a == b AND a == length(c) 312 ) FROM abc;313 }314 } {1}315}316SQL {317 DELETE FROM abc WHERE oid IN (SELECT oid FROM abc ORDER BY random() LIMIT 5);318}319TEST 7 {320 do_test $testid {321 execsql {SELECT count(*) FROM abc}322 } {89}323 do_test $testid {324 execsql {325 SELECT min(326 (oid == a) AND 'String value ' || a == b AND a == length(c) 327 ) FROM abc;328 }329 } {1}330}331SQL {332 DELETE FROM abc WHERE oid IN (SELECT oid FROM abc ORDER BY random() LIMIT 5);333}334TEST 9 {335 do_test $testid {336 execsql {SELECT count(*) FROM abc}337 } {84}338 do_test $testid {339 execsql {340 SELECT min(341 (oid == a) AND 'String value ' || a == b AND a == length(c) 342 ) FROM abc;343 }344 } {1}345}346 347set padding [string repeat X 500]348PREP [subst {349 DROP TABLE abc;350 CREATE TABLE abc(a PRIMARY KEY, padding, b, c);351 INSERT INTO abc VALUES(0, '$padding', 2, 2);352 INSERT INTO abc VALUES(3, '$padding', 5, 5);353 INSERT INTO abc VALUES(6, '$padding', 8, 8);354}]355 356TEST 10 {357 do_test $testid {358 execsql {SELECT a, b, c FROM abc}359 } {0 2 2 3 5 5 6 8 8}360}361 362SQL {BEGIN;}363SQL {INSERT INTO abc VALUES(9, 'XXXXX', 11, 12);}364TEST_AUTOCOMMIT 11 0365SQL -norollback {UPDATE abc SET a = a + 1, c = c + 1;}366TEST_AUTOCOMMIT 12 0367SQL {DELETE FROM abc WHERE a = 10;}368TEST_AUTOCOMMIT 13 0369SQL {COMMIT;}370 371TEST 14 {372 do_test $testid.1 {373 sqlite3_get_autocommit $::DB374 } {1}375 do_test $testid.2 {376 execsql {SELECT a, b, c FROM abc}377 } {1 2 3 4 5 6 7 8 9}378}379 380PREP [subst {381 DROP TABLE abc;382 CREATE TABLE abc(a, padding, b, c);383 INSERT INTO abc VALUES(1, '$padding', 2, 3);384 INSERT INTO abc VALUES(4, '$padding', 5, 6);385 INSERT INTO abc VALUES(7, '$padding', 8, 9);386 CREATE INDEX abc_i ON abc(a, padding, b, c);387}]388 389TEST 15 {390 db eval {PRAGMA cache_size = 10}391}392 393SQL {BEGIN;}394SQL -norllbck {INSERT INTO abc (oid, a, padding, b, c) SELECT NULL, * FROM abc}395TEST 16 {396 do_test $testid {397 execsql {SELECT a, count(*) FROM abc GROUP BY a;}398 } {1 2 4 2 7 2}399}400SQL -norllbck {INSERT INTO abc (oid, a, padding, b, c) SELECT NULL, * FROM abc}401TEST 17 {402 do_test $testid {403 execsql {SELECT a, count(*) FROM abc GROUP BY a;}404 } {1 4 4 4 7 4}405}406SQL -norllbck {INSERT INTO abc (oid, a, padding, b, c) SELECT NULL, * FROM abc}407TEST 18 {408 do_test $testid {409 execsql {SELECT a, count(*) FROM abc GROUP BY a;}410 } {1 8 4 8 7 8}411}412SQL -norllbck {INSERT INTO abc (oid, a, padding, b, c) SELECT NULL, * FROM abc}413TEST 19 {414 do_test $testid {415 execsql {SELECT a, count(*) FROM abc GROUP BY a;}416 } {1 16 4 16 7 16}417}418SQL {COMMIT;}419TEST 21 {420 do_test $testid {421 execsql {SELECT a, count(*) FROM abc GROUP BY a;}422 } {1 16 4 16 7 16}423}424 425SQL {BEGIN;}426SQL {DELETE FROM abc WHERE oid %2}427TEST 22 {428 do_test $testid {429 execsql {SELECT a, count(*) FROM abc GROUP BY a;}430 } {1 8 4 8 7 8}431}432SQL {DELETE FROM abc}433TEST 23 {434 do_test $testid {435 execsql {SELECT * FROM abc}436 } {}437}438SQL {ROLLBACK;}439TEST 24 {440 do_test $testid {441 execsql {SELECT a, count(*) FROM abc GROUP BY a;}442 } {1 16 4 16 7 16}443}444 445# Test some schema modifications inside of a transaction. These should all446# cause transaction rollback if they fail. Also query a view, to cover a bit447# more code.448#449PREP {DROP VIEW abc_v;}450TEST 25 {451 do_test $testid {452 execsql {453 SELECT name, tbl_name FROM sqlite_master;454 }455 } {abc abc abc_i abc}456}457SQL {BEGIN;}458SQL {CREATE TABLE def(d, e, f);}459SQL {CREATE TABLE ghi(g, h, i);}460TEST 26 {461 do_test $testid {462 execsql {463 SELECT name, tbl_name FROM sqlite_master;464 }465 } {abc abc abc_i abc def def ghi ghi}466}467SQL {CREATE VIEW v1 AS SELECT * FROM def, ghi}468SQL {CREATE UNIQUE INDEX ghi_i1 ON ghi(g);}469TEST 27 {470 do_test $testid {471 execsql {472 SELECT name, tbl_name FROM sqlite_master;473 }474 } {abc abc abc_i abc def def ghi ghi v1 v1 ghi_i1 ghi}475}476SQL {INSERT INTO def VALUES('a', 'b', 'c')}477SQL {INSERT INTO def VALUES(1, 2, 3)}478SQL -norollback {INSERT INTO ghi SELECT * FROM def}479TEST 28 {480 do_test $testid {481 execsql {482 SELECT * FROM def, ghi WHERE d = g;483 }484 } {a b c a b c 1 2 3 1 2 3}485}486SQL {COMMIT}487TEST 29 {488 do_test $testid {489 execsql {490 SELECT * FROM v1 WHERE d = g;491 }492 } {a b c a b c 1 2 3 1 2 3}493}494 495# Test a simple multi-file transaction 496#497forcedelete test2.db498ifcapable attach {499 SQL {ATTACH 'test2.db' AS aux;}500 SQL {BEGIN}501 SQL {CREATE TABLE aux.tbl2(x, y, z)}502 SQL {INSERT INTO tbl2 VALUES(1, 2, 3)}503 SQL {INSERT INTO def VALUES(4, 5, 6)}504 TEST 30 {505 do_test $testid {506 execsql {507 SELECT * FROM tbl2, def WHERE d = x;508 }509 } {1 2 3 1 2 3}510 }511 SQL {COMMIT}512 TEST 31 {513 do_test $testid {514 execsql {515 SELECT * FROM tbl2, def WHERE d = x;516 }517 } {1 2 3 1 2 3}518 }519}520 521# Test what happens when a malloc() fails while there are other active522# statements. This changes the way sqlite3VdbeHalt() works.523TEST 32 {524 if {![info exists ::STMT32]} {525 set sql "SELECT name FROM sqlite_master"526 set ::STMT32 [sqlite3_prepare $::DB $sql -1 DUMMY]527 do_test $testid {528 sqlite3_step $::STMT32529 } {SQLITE_ROW}530 }531}532SQL BEGIN533TEST 33 { 534 do_test $testid {535 execsql {SELECT * FROM ghi}536 } {a b c 1 2 3}537}538SQL -norollback { 539 -- There is a unique index on ghi(g), so this statement may not cause540 -- an automatic ROLLBACK. Hence the "-norollback" switch.541 INSERT INTO ghi SELECT '2'||g, h, i FROM ghi;542}543TEST 34 {544 if {[info exists ::STMT32]} {545 do_test $testid {546 sqlite3_finalize $::STMT32547 } {SQLITE_OK}548 unset ::STMT32549 }550}551SQL COMMIT552 553#554# End of test program declaration555#--------------------------------------------------------------------------556 557proc run_test {arglist iRepeat {pcstart 0} {iFailStart 1}} {558 if {[llength $arglist] %2} {559 error "Uneven number of arguments to TEST"560 }561 562 for {set i 0} {$i < $pcstart} {incr i} {563 set k2 [lindex $arglist [expr {2 * $i}]]564 set v2 [lindex $arglist [expr {2 * $i + 1}]]565 set ac [sqlite3_get_autocommit $::DB] ;# Auto-Commit566 switch -- $k2 {567 -sql {db eval [lindex $v2 2]}568 -prep {db eval $v2}569 -debug {eval $v2}570 }571 set nac [sqlite3_get_autocommit $::DB] ;# New Auto-Commit 572 if {$ac && !$nac} {set begin_pc $i}573 }574 575 db rollback_hook [list incr ::rollback_hook_count]576 577 set iFail $iFailStart578 set pc $pcstart579 while {$pc*2 < [llength $arglist]} {580 # Fetch the current instruction type and payload.581 set k [lindex $arglist [expr {2 * $pc}]]582 set v [lindex $arglist [expr {2 * $pc + 1}]]583 584 # Id of this iteration:585 set iterid "pc=$pc.iFail=$iFail$k"586 587 switch -- $k {588 589 -test { 590 foreach {id script} $v {}591 set testid "malloc3-(test $id).$iterid"592 eval $script593 incr pc594 }595 596 -sql {597 set ::rollback_hook_count 0598 599 set id [lindex $v 0]600 set testid "malloc3-(integrity $id).$iterid"601 602 set ac [sqlite3_get_autocommit $::DB] ;# Auto-Commit603 sqlite3_memdebug_fail $iFail -repeat 0604 set rc [catch {db eval [lindex $v 2]} msg] ;# True error occurs605 set nac [sqlite3_get_autocommit $::DB] ;# New Auto-Commit 606 607 if {$rc != 0 && $nac && !$ac} {608 # Before [db eval] the auto-commit flag was clear. Now it609 # is set. Since an error occurred we assume this was not a610 # commit - therefore a rollback occurred. Check that the611 # rollback-hook was invoked.612 do_test malloc3-rollback_hook_count.$iterid {613 set ::rollback_hook_count614 } {1}615 }616 617 set nFail [sqlite3_memdebug_fail -1 -benigncnt nBenign]618 if {$rc == 0} {619 # Successful execution of sql. The number of failed malloc()620 # calls should be equal to the number of benign failures.621 # Otherwise a malloc() failed and the error was not reported.622 # 623 set expr {$nFail!=$nBenign}624 if {[expr $expr]} {625 error "Unreported malloc() failure, test \"$testid\", $expr"626 }627 628 if {$ac && !$nac} {629 # Before the [db eval] the auto-commit flag was set, now it630 # is clear. We can deduce that a "BEGIN" statement has just631 # been successfully executed.632 set begin_pc $pc633 } 634 635 incr pc636 set iFail 1637 integrity_check $testid638 } elseif {[regexp {.*out of memory} $msg] || [db errorcode] == 3082} {639 # Out of memory error, as expected.640 #641 integrity_check $testid642 incr iFail643 if {$nac && !$ac} {644 if {![lindex $v 1] && [db errorcode] != 3082} {645 # error "Statement \"[lindex $v 2]\" caused a rollback"646 }647 648 for {set i $begin_pc} {$i < $pc} {incr i} {649 set k2 [lindex $arglist [expr {2 * $i}]]650 set v2 [lindex $arglist [expr {2 * $i + 1}]]651 set catchupsql ""652 switch -- $k2 {653 -sql {set catchupsql [lindex $v2 2]}654 -prep {set catchupsql $v2}655 }656 db eval $catchupsql657 }658 }659 } else {660 error $msg661 }662 663 # back up to the previous "-test" block.664 while {[lindex $arglist [expr {2 * ($pc - 1)}]] == "-test"} {665 incr pc -1666 }667 }668 669 -prep {670 db eval $v671 incr pc672 }673 674 -debug {675 eval $v676 incr pc677 }678 679 default { error "Unknown switch: $k" }680 }681 }682}683 684# Turn off the Tcl interface's prepared statement caching facility. Then685# run the tests with "persistent" malloc failures.686sqlite3_extended_result_codes db 1687db cache size 0688run_test $::run_test_script 1689 690# Close and reopen the db.691db close692forcedelete test.db test.db-journal test2.db test2.db-journal693sqlite3 db test.db694sqlite3_extended_result_codes db 1695set ::DB [sqlite3_connection_pointer db]696 697# Turn off the Tcl interface's prepared statement caching facility in698# the new connnection. Then run the tests with "transient" malloc failures.699db cache size 0700run_test $::run_test_script 0701 702sqlite3_memdebug_fail -1703finish_test704