50 Commits

Author SHA1 Message Date
Valentin Lorentz
4da1291876 URL: Lazily deserialize records from the end in @last
Before this commit, the plugin first fetched a list of all
(deserialized) records in a list, then reversed the list, and iterated
on the reverse list.
This proved to be slow, with most of the time being spent in
`dbi.DB._newRecord` (which essentially deserializes one list of CSV).

After this commit, the list is reversed first, then the plugin iterates
on its generator, which calls `_newRecord` on records as they are
requested.

This means that when there are many URLs in the database, `@last` does
not need to waste time deserializing most records, when the result is
near the end (and if the result is the first record, then it does
exactly as much work as before).
2022-10-30 20:46:35 +01:00
Valentin Lorentz
a6aa5530dd Ensure files written with AtomicFile are read in UTF8
With some locale configurations (not that uncommon on CentOS), open() may
default to non-UTF8 encodings (eg. ANSI_X3.4-1968).

This is usually not an issue, because we use open() both for writing and
reading. However, AtomicFile implicitly enforces UTF8; which needs to be
mirrored when reading.
2022-10-06 09:49:42 +02:00
Valentin Lorentz
63eb6672ea Revert generic 'The Limnoria Contributors' in copyright notices
This commit reverts db7ef3f02517f9f2a3c56829a22b9fad3c36e374
(though it keeps the year updates)

After discussion with several people, it seems better to mention
copyright owners explicitly. eg. https://reuse.software/faq/#vcs-copyright
explains the issue of using VCSs to track copyright.

As db7ef3f02517f9f2a3c56829a22b9fad3c36e374 only replaced mentions
of my name with 'The Limnoria Contributors', this commit only needs
to undo that + add one person who contributed to setup.py.
2021-10-17 09:57:55 +02:00
Valentin Lorentz
db7ef3f025 all: Add generic 'The Limnoria Contributors' to copyright notices.
No need to bother with details (that are all outdated / out of sync
anyway), just look up the git history.
2021-08-01 21:54:49 +02:00
Valentin Lorentz
912d2e1538 core: Apply some suggestions of pyflakes. 2015-08-31 18:19:22 +02:00
Valentin Lorentz
c3a2c800f1 Remove need for 2to3. 2015-08-11 16:50:23 +02:00
Valentin Lorentz
f305ff3c02 Remove fix_basestring. 2015-08-10 20:24:11 +02:00
Valentin Lorentz
be118c3338 Remove use of fix_dict fixer. 2015-08-08 23:08:49 +02:00
Valentin Lorentz
bb7db3ab21 Continue accelerating the 2to3 step (remove fix_except). 2014-01-20 15:49:15 +01:00
Valentin Lorentz
2fda69b4d6 Continue accelerating the 2to3 step (remove fix_raise). 2014-01-20 15:43:55 +01:00
Valentin Lorentz
b44a08d2bd Fix previous merge (again). 2014-01-03 17:48:30 +00:00
nyuszika7h
0e480b4e52 Fix file descriptor leaks 2014-01-03 17:44:01 +01:00
Valentin Lorentz
40675ffdfa Merge remote-tracking branch 'supybot/master' into testing
Conflicts:
	plugins/Admin/plugin.py
	plugins/BadWords/config.py
	plugins/Filter/plugin.py
	plugins/Google/plugin.py
	plugins/Math/plugin.py
	plugins/Misc/plugin.py
	plugins/Note/plugin.py
	plugins/RSS/plugin.py
	plugins/Seen/test.py
	plugins/ShrinkUrl/config.py
	plugins/ShrinkUrl/plugin.py
	plugins/ShrinkUrl/test.py
	plugins/Status/plugin.py
	plugins/String/config.py
	plugins/String/plugin.py
	plugins/Time/plugin.py
	plugins/Todo/plugin.py
	plugins/Web/plugin.py
	plugins/__init__.py
	scripts/supybot-botchk
	setup.py
	src/__init__.py
	src/callbacks.py
	src/commands.py
	src/conf.py
	src/drivers/Socket.py
	src/ircdb.py
	src/irclib.py
	src/ircutils.py
	src/questions.py
	src/registry.py
	src/schedule.py
	src/test.py
	src/utils/file.py
	src/utils/gen.py
	src/utils/net.py
	src/utils/web.py
	src/world.py
2013-08-24 11:28:29 +02:00
James McCoy
cef93a6cfd Use relative imports for all packages under the supybot namespace
Signed-off-by: James McCoy <jamessan@users.sourceforge.net>
2012-09-17 22:12:11 -04:00
Valentin Lorentz
535593d555 src/dbi.py: Use open() instead of file(). 2012-08-05 16:56:07 +02:00
Valentin Lorentz
14200e002a Rename cdb.open to cdb.open_db and open cdb files in binary mode. 2012-08-05 15:47:30 +02:00
Valentin Lorentz
14609b3bb2 Add self as an argument to methos in src/dbi.py 2012-05-22 21:15:58 -04:00
Valentin Lorentz
1fe4cef61b Import os before using it. 2012-05-22 21:14:42 -04:00
Valentin Lorentz
af460596d3 Add self as an argument to methos in src/dbi.py 2012-05-02 20:50:06 +02:00
Valentin Lorentz
776ce96d52 Import os before using it. 2012-05-02 20:48:49 +02:00
James Vega
a3e4fc5b1d Change the modeline to use softtabstop instead of tabstop. 2006-02-11 15:52:51 +00:00
James Vega
9971e991fe Fix the modelines. 2005-03-23 20:07:45 +00:00
Ali Afshar
4404cdc51a dbi cdb mapping trying to use a non-existent variable. 2005-03-21 15:53:13 +00:00
Jeremy Fincher
cb05dc8484 Fixed a typo in a variable name. 2005-02-01 10:13:23 +00:00
James Vega
ec045afd9c Removed extraneous whitespace, unused imports, and converted some sets.Set() calls to set(). 2005-01-31 15:24:36 +00:00
James Vega
0fde2393e6 Update usage of utils.iter functions. 2005-01-31 15:22:48 +00:00
James Vega
0a89623b70 Update a couple more imports for the utils split up and fix.py removal. 2005-01-30 15:39:39 +00:00
Jeremy Fincher
5fd6bbb52d Completely restructured our utils modules.
Tons of changes.  Here's the summary of things that matter most:

* There is no more supybot.fix.
* There is no more supybot.webutils; now there is supybot.utils.web.
  * It's no longer webutils.WebError, but just utils.web.Error.
* You shouldn't import itertools, ideally, but instead import utils.iter.
* No more using imap/ifilter in commands unless absolutely necessary.  It's
  premature optimization and annoying.
* utils.str.format isn't quite ready yet, but will be soon.  That'll be the
  next big thing to fix in our code.
2005-01-27 06:59:08 +00:00
Jeremy Fincher
5019d10a1b Updated license years. 2005-01-19 13:33:05 +00:00
Jeremy Fincher
8ca625d339 Initial import. 2005-01-19 13:14:38 +00:00
James Vega
dbfec8afb9 Add supybot module 2007-04-22 20:10:26 +00:00
Jeremy Fincher
4624115909 No need to makeBackupIfSmaller in databases. That's more for configuration files. 2005-01-14 14:46:52 +00:00
Jeremy Fincher
cbba7c6115 Fixed a potential bug. 2004-12-02 04:51:58 +00:00
Jeremy Fincher
9021c5e150 DirMapping is incomplete, but I need some feature additions for DB. 2004-12-01 21:47:02 +00:00
Jeremy Fincher
54b56a8a53 Raise the proper exception. 2004-11-30 04:42:20 +00:00
Jeremy Fincher
744944d8b2 Wrong variable name. 2004-10-29 19:27:25 +00:00
Jeremy Fincher
6ba2120f5f Optimization. 2004-10-04 16:11:08 +00:00
Jeremy Fincher
e3c9464634 Return None if there's nothing in the database. 2004-10-01 21:25:26 +00:00
Jeremy Fincher
0e5cd79a1b eXXXtra eXXXtra, read all about it. 2004-10-01 08:22:48 +00:00
Jeremy Fincher
4cc1e4ba04 Abstracted for some future overriding. 2004-09-30 04:15:32 +00:00
Jeremy Fincher
e01a9d3bc5 Forgot to vacuum, momma's gonna kill me! 2004-09-28 20:09:39 +00:00
Jeremy Fincher
5753195f45 Changed dbi.Record not to use a metaclass. 2004-09-24 20:05:34 +00:00
James Vega
50973a10fb Remove #! lines. 2004-09-20 03:01:50 +00:00
Jeremy Fincher
acf0a3f3e5 Augmented the add method for Note. 2004-08-24 04:09:15 +00:00
Jeremy Fincher
fbb348b6e7 Copyright updates. Again, more is left to be done, but this is my part. 2004-08-23 13:14:06 +00:00
James Vega
72f36efc18 Fix some problems in Infobot.py's sqlite implementation and genericize
some other errors to better fit in with db abstraction
2004-08-17 20:01:23 +00:00
James Vega
46c71a7521 Add dbi.NoRecordError and update Quotes to use it 2004-08-16 17:34:58 +00:00
Jeremy Fincher
a1a703df32 Let's go to repr. Beware, people, this may invalidate certain databases. 2004-08-12 07:41:05 +00:00
Jeremy Fincher
12ee483f7a Slight bugfixes; returning from a remove is unpossible. 2004-08-11 05:48:55 +00:00
Jeremy Fincher
e5e98fdc2f Initial checkin. 2004-08-11 05:14:15 +00:00