« back to all changes in this revision
Viewing changes to plugin/myisam/mf_keycache.cc
- browse files at revision 2414
- compare with another revision
- compare with revision 2396
- download diff
- download tarball
- view history from revision 2414
- Committer: Mark Atwood
- Date: 2011-09-14 03:30:42 UTC
- mfrom: (2409.2.6 refactor7)
- Revision ID: me@mark.atwood.name-20110914033042-2u0s8foaigvf62g2
merge lp:~olafvdspek/drizzle/refactor7
- files removed:
- plugin/myisam/keycache.h
- files modified:
- drizzled/plugin/storage_engine.cc
added
removed
plugin/myisam/mf_keycache.cc
1
/* Copyright (C) 2000 MySQL AB
2
3
This program is free software; you can redistribute it and/or modify
4
it under the terms of the GNU General Public License as published by
5
the Free Software Foundation; version 2 of the License.
6
7
This program is distributed in the hope that it will be useful,
8
but WITHOUT ANY WARRANTY; without even the implied warranty of
9
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
10
GNU General Public License for more details.
11
12
You should have received a copy of the GNU General Public License
13
along with this program; if not, write to the Free Software
14
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
15
16
/*
17
These functions handle keyblock cacheing for ISAM and MyISAM tables.
18
19
One cache can handle many files.
20
It must contain buffers of the same blocksize.
21
init_key_cache() should be used to init cache handler.
22
23
The free list (free_block_list) is a stack like structure.
24
When a block is freed by free_block(), it is pushed onto the stack.
25
When a new block is required it is first tried to pop one from the stack.
26
If the stack is empty, it is tried to get a never-used block from the pool.
27
If this is empty too, then a block is taken from the LRU ring, flushing it
28
to disk, if neccessary. This is handled in find_key_block().
29
With the new free list, the blocks can have three temperatures:
30
hot, warm and cold (which is free). This is remembered in the block header
31
by the enum BLOCK_TEMPERATURE temperature variable. Remembering the
32
temperature is neccessary to correctly count the number of warm blocks,
33
which is required to decide when blocks are allowed to become hot. Whenever
34
a block is inserted to another (sub-)chain, we take the old and new
35
temperature into account to decide if we got one more or less warm block.
36
blocks_unused is the sum of never used blocks in the pool and of currently
37
free blocks. blocks_used is the number of blocks fetched from the pool and
38
as such gives the maximum number of in-use blocks at any time.
39
40
Key Cache Locking
41
=================
42
43
All key cache locking is done with a single mutex per key cache:
44
keycache->cache_lock. This mutex is locked almost all the time
45
when executing code in this file (mf_keycache.c).
46
However it is released for I/O and some copy operations.
47
48
The cache_lock is also released when waiting for some event. Waiting
49
and signalling is done via condition variables. In most cases the
50
thread waits on its thread->suspend condition variable. Every thread
51
has a my_thread_var structure, which contains this variable and a
52
'*next' and '**prev' pointer. These pointers are used to insert the
53
thread into a wait queue.
54
55
A thread can wait for one block and thus be in one wait queue at a
56
time only.
57
58
Before starting to wait on its condition variable with
59
pthread_cond_wait(), the thread enters itself to a specific wait queue
60
with link_into_queue() (double linked with '*next' + '**prev') or
61
wait_on_queue() (single linked with '*next').
62
63
Another thread, when releasing a resource, looks up the waiting thread
64
in the related wait queue. It sends a signal with
65
pthread_cond_signal() to the waiting thread.
66
67
NOTE: Depending on the particular wait situation, either the sending
68
thread removes the waiting thread from the wait queue with
69
unlink_from_queue() or release_whole_queue() respectively, or the waiting
70
thread removes itself.
71
72
There is one exception from this locking scheme when one thread wants
73
to reuse a block for some other address. This works by first marking
74
the block reserved (status= BLOCK_IN_SWITCH) and then waiting for all
75
threads that are reading the block to finish. Each block has a
76
reference to a condition variable (condvar). It holds a reference to
77
the thread->suspend condition variable for the waiting thread (if such
78
a thread exists). When that thread is signaled, the reference is
79
cleared. The number of readers of a block is registered in
80
block->hash_link->requests. See wait_for_readers() / remove_reader()
81
for details. This is similar to the above, but it clearly means that
82
only one thread can wait for a particular block. There is no queue in
83
this case. Strangely enough block->convar is used for waiting for the
84
assigned hash_link only. More precisely it is used to wait for all
85
requests to be unregistered from the assigned hash_link.
86
87
The resize_queue serves two purposes:
88
1. Threads that want to do a resize wait there if in_resize is set.
89
This is not used in the server. The server refuses a second resize
90
request if one is already active. keycache->in_init is used for the
91
synchronization. See set_var.cc.
92
2. Threads that want to access blocks during resize wait here during
93
the re-initialization phase.
94
When the resize is done, all threads on the queue are signalled.
95
Hypothetical resizers can compete for resizing, and read/write
96
requests will restart to request blocks from the freshly resized
97
cache. If the cache has been resized too small, it is disabled and
98
'can_be_used' is false. In this case read/write requests bypass the
99
cache. Since they increment and decrement 'cnt_for_resize_op', the
100
next resizer can wait on the queue 'waiting_for_resize_cnt' until all
101
I/O finished.
102
*/
103
104
#include <config.h>
105
#include <drizzled/error.h>
106
#include <drizzled/internal/my_sys.h>
107
#include "keycache.h"
108
#include <drizzled/internal/m_string.h>
109
#include <drizzled/internal/my_bit.h>
110
#include <errno.h>
111
#include <stdarg.h>
112
113
using namespace drizzled;
114
115
/*
116
Some compilation flags have been added specifically for this module
117
to control the following:
118
- not to let a thread to yield the control when reading directly
119
from key cache, which might improve performance in many cases;
120
to enable this add:
121
#define SERIALIZED_READ_FROM_CACHE
122
- to set an upper bound for number of threads simultaneously
123
using the key cache; this setting helps to determine an optimal
124
size for hash table and improve performance when the number of
125
blocks in the key cache much less than the number of threads
126
accessing it;
127
to set this number equal to <N> add
128
#define MAX_THREADS <N>
129
130
Example of the settings:
131
#define SERIALIZED_READ_FROM_CACHE
132
#define MAX_THREADS 100
133
*/
134
135
#define STRUCT_PTR(TYPE, MEMBER, a) \
136
(TYPE *) ((char *) (a) - offsetof(TYPE, MEMBER))
137
138
/* types of condition variables */
139
#define COND_FOR_REQUESTED 0
140
#define COND_FOR_SAVED 1
141
142
typedef pthread_cond_t KEYCACHE_CONDVAR;
143
144
/* descriptor of the page in the key cache block buffer */
145
struct st_keycache_page
146
{
147
int file; /* file to which the page belongs to */
148
internal::my_off_t filepos; /* position of the page in the file */
149
};
150
151
/* element in the chain of a hash table bucket */
152
struct st_hash_link
153
{
154
struct st_hash_link *next, **prev; /* to connect links in the same bucket */
155
struct st_block_link *block; /* reference to the block for the page: */
156
int file; /* from such a file */
157
internal::my_off_t diskpos; /* with such an offset */
158
uint32_t requests; /* number of requests for the page */
159
};
160
161
/* simple states of a block */
162
#define BLOCK_ERROR 1 /* an error occured when performing file i/o */
163
#define BLOCK_READ 2 /* file block is in the block buffer */
164
#define BLOCK_IN_SWITCH 4 /* block is preparing to read new page */
165
#define BLOCK_REASSIGNED 8 /* blk does not accept requests for old page */
166
#define BLOCK_IN_FLUSH 16 /* block is selected for flush */
167
#define BLOCK_CHANGED 32 /* block buffer contains a dirty page */
168
#define BLOCK_IN_USE 64 /* block is not free */
169
#define BLOCK_IN_EVICTION 128 /* block is selected for eviction */
170
#define BLOCK_IN_FLUSHWRITE 256 /* block is in write to file */
171
#define BLOCK_FOR_UPDATE 512 /* block is selected for buffer modification */
172
173
/* page status, returned by find_key_block */
174
#define PAGE_READ 0
175
#define PAGE_TO_BE_READ 1
176
#define PAGE_WAIT_TO_BE_READ 2
177
178
/* block temperature determines in which (sub-)chain the block currently is */
179
enum BLOCK_TEMPERATURE { BLOCK_COLD /*free*/ , BLOCK_WARM , BLOCK_HOT };
180
181
/* key cache block */
182
struct st_block_link
183
{
184
struct st_block_link
185
*next_used, **prev_used; /* to connect links in the LRU chain (ring) */
186
struct st_block_link
187
*next_changed, **prev_changed; /* for lists of file dirty/clean blocks */
188
struct st_hash_link *hash_link; /* backward ptr to referring hash_link */
189
KEYCACHE_WQUEUE wqueue[2]; /* queues on waiting requests for new/old pages */
190
uint32_t requests; /* number of requests for the block */
191
unsigned char *buffer; /* buffer for the block page */
192
uint32_t offset; /* beginning of modified data in the buffer */
193
uint32_t length; /* end of data in the buffer */
194
uint32_t status; /* state of the block */
195
enum BLOCK_TEMPERATURE temperature; /* block temperature: cold, warm, hot */
196
uint32_t hits_left; /* number of hits left until promotion */
197
uint64_t last_hit_time; /* timestamp of the last hit */
198
KEYCACHE_CONDVAR *condvar; /* condition variable for 'no readers' event */
199
};
200
201
#define FLUSH_CACHE 2000 /* sort this many blocks at once */
202
203
#define KEYCACHE_HASH(f, pos) \
204
(((uint32_t) ((pos) / keycache->key_cache_block_size) + \
205
(uint32_t) (f)) & (keycache->hash_entries-1))
206
#define FILE_HASH(f) ((uint) (f) & (CHANGED_BLOCKS_HASH-1))
207
208
209
#define keycache_pthread_cond_wait(A,B) (void)A;
210
#define keycache_pthread_mutex_lock(A) (void)A;
211
#define keycache_pthread_mutex_unlock(A) (void)A;
212
#define keycache_pthread_cond_signal(A) (void)A;
213
214
static inline uint32_t next_power(uint32_t value)
215
{
216
return my_round_up_to_next_power(value) << 1;
217
}
218
219
220
/*
221
Initialize a key cache
222
223
SYNOPSIS
224
init_key_cache()
225
keycache pointer to a key cache data structure
226
key_cache_block_size size of blocks to keep cached data
227
use_mem total memory to use for the key cache
228
division_limit division limit (may be zero)
229
age_threshold age threshold (may be zero)
230
231
RETURN VALUE
232
number of blocks in the key cache, if successful,
233
0 - otherwise.
234
235
NOTES.
236
if keycache->key_cache_inited != 0 we assume that the key cache
237
is already initialized. This is for now used by myisamchk, but shouldn't
238
be something that a program should rely on!
239
240
It's assumed that no two threads call this function simultaneously
241
referring to the same key cache handle.
242
243
*/
244
245
int init_key_cache(KEY_CACHE *keycache, uint32_t key_cache_block_size,
246
size_t use_mem, uint32_t division_limit,
247
uint32_t age_threshold)
248
{
249
(void)keycache;
250
(void)key_cache_block_size;
251
(void)use_mem;
252
(void)division_limit;
253
(void)age_threshold;
254
memset(keycache, 0, sizeof(KEY_CACHE));
255
256
return 0;
257
}
258
259
260
/*
261
Remove key_cache from memory
262
263
SYNOPSIS
264
end_key_cache()
265
keycache key cache handle
266
cleanup Complete free (Free also mutex for key cache)
267
268
RETURN VALUE
269
none
270
*/
271
272
void end_key_cache(KEY_CACHE *keycache, bool cleanup)
273
{
274
(void)keycache;
275
(void)cleanup;
276
} /* end_key_cache */
277
278
279
/*
280
Add a hash link to a bucket in the hash_table
281
*/
282
283
static inline void link_hash(HASH_LINK **start, HASH_LINK *hash_link)
284
{
285
if (*start)
286
(*start)->prev= &hash_link->next;
287
hash_link->next= *start;
288
hash_link->prev= start;
289
*start= hash_link;
290
}
291
292
293
/*
294
Read a block of data from a cached file into a buffer;
295
296
SYNOPSIS
297
298
key_cache_read()
299
keycache pointer to a key cache data structure
300
file handler for the file for the block of data to be read
301
filepos position of the block of data in the file
302
level determines the weight of the data
303
buff buffer to where the data must be placed
304
length length of the buffer
305
block_length length of the block in the key cache buffer
306
return_buffer return pointer to the key cache buffer with the data
307
308
RETURN VALUE
309
Returns address from where the data is placed if sucessful, 0 - otherwise.
310
311
NOTES.
312
The function ensures that a block of data of size length from file
313
positioned at filepos is in the buffers for some key cache blocks.
314
Then the function either copies the data into the buffer buff, or,
315
if return_buffer is true, it just returns the pointer to the key cache
316
buffer with the data.
317
Filepos must be a multiple of 'block_length', but it doesn't
318
have to be a multiple of key_cache_block_size;
319
*/
320
321
unsigned char *key_cache_read(KEY_CACHE *keycache,
322
int file, internal::my_off_t filepos, int level,
323
unsigned char *buff, uint32_t length,
324
uint32_t block_length,
325
int return_buffer)
326
{
327
(void)block_length;
328
(void)return_buffer;
329
(void)level;
330
int error=0;
331
unsigned char *start= buff;
332
333
assert (! keycache->key_cache_inited);
334
335
if (!pread(file, (unsigned char*) buff, length, filepos))
336
error= 1;
337
return(error ? (unsigned char*) 0 : start);
338
}
339
340
341
/*
342
Insert a block of file data from a buffer into key cache
343
344
SYNOPSIS
345
key_cache_insert()
346
keycache pointer to a key cache data structure
347
file handler for the file to insert data from
348
filepos position of the block of data in the file to insert
349
level determines the weight of the data
350
buff buffer to read data from
351
length length of the data in the buffer
352
353
NOTES
354
This is used by MyISAM to move all blocks from a index file to the key
355
cache
356
357
RETURN VALUE
358
0 if a success, 1 - otherwise.
359
*/
360
361
int key_cache_insert(KEY_CACHE *keycache,
362
int file, internal::my_off_t filepos, int level,
363
unsigned char *buff, uint32_t length)
364
{
365
(void)file;
366
(void)filepos;
367
(void)level;
368
(void)buff;
369
(void)length;
370
371
assert (!keycache->key_cache_inited);
372
return 0;
373
}
374
375
376
/*
377
Write a buffer into a cached file.
378
379
SYNOPSIS
380
381
key_cache_write()
382
keycache pointer to a key cache data structure
383
file handler for the file to write data to
384
filepos position in the file to write data to
385
level determines the weight of the data
386
buff buffer with the data
387
length length of the buffer
388
dont_write if is 0 then all dirty pages involved in writing
389
should have been flushed from key cache
390
391
RETURN VALUE
392
0 if a success, 1 - otherwise.
393
394
NOTES.
395
The function copies the data of size length from buff into buffers
396
for key cache blocks that are assigned to contain the portion of
397
the file starting with position filepos.
398
It ensures that this data is flushed to the file if dont_write is false.
399
Filepos must be a multiple of 'block_length', but it doesn't
400
have to be a multiple of key_cache_block_size;
401
402
dont_write is always true in the server (info->lock_type is never F_UNLCK).
403
*/
404
405
int key_cache_write(KEY_CACHE *keycache,
406
int file, internal::my_off_t filepos, int level,
407
unsigned char *buff, uint32_t length,
408
uint32_t block_length,
409
int dont_write)
410
{
411
(void)block_length;
412
(void)level;
413
int error=0;
414
415
if (!dont_write)
416
{
417
/* Not used in the server. */
418
/* Force writing from buff into disk. */
419
if (pwrite(file, buff, length, filepos) == 0)
420
return(1);
421
}
422
423
assert (!keycache->key_cache_inited);
424
425
/* Key cache is not used */
426
if (dont_write)
427
{
428
/* Used in the server. */
429
if (pwrite(file, (unsigned char*) buff, length, filepos) == 0)
430
error=1;
431
}
432
433
return(error);
434
}
435
436
437
/*
438
Flush all blocks for a file to disk
439
440
SYNOPSIS
441
442
flush_key_blocks()
443
keycache pointer to a key cache data structure
444
file handler for the file to flush to
445
flush_type type of the flush
446
447
RETURN
448
0 ok
449
1 error
450
*/
451
452
int flush_key_blocks(KEY_CACHE *keycache,
453
int file, enum flush_type type)
454
{
455
(void)file;
456
(void)type;
457
assert (!keycache->key_cache_inited);
458
return 0;
459
}
Loggerhead is a web-based interface for Breezy
Version: 2.0.1