twitter: comp.lang.python - 25 new messages in 11 topics

comp.lang.python
http://groups.google.com/group/comp.lang.python?hl=en

comp.lang.python@googlegroups.com

Today's topics:

==============================================================================
TOPIC: Python bindings tutorial
http://groups.google.com/group/comp.lang.python/t/73dcf6c0b97465aa?hl=en
==============================================================================

== 1 of 1 ==
Date: Fri, Mar 19 2010 12:01 am
From: "Alf P. Steinbach"

* Tim Roberts:
> Dave Angel <davea@ieee.org> wrote:
>> There's no real reason parts of an exe cannot be exported, same as a
>> dll. They are in fact the same structure. And in fact many other files
>> in the Windows environment are also the same structure, from fonts to ocx's

This is a bit off-topic, but your explanation is incorrect in some key respects,
so (no offense) to avoid readers getting an incorrect impression:

> Well, there IS a fundamental difference. EXEs and DLLs and the like do all
> have the same format. They all have a "transfer address"

Commonly called an "entry point".

This is the term you need to know about when e.g. linking object files.

>, where execution
> begins. That's the key problem. With a DLL, the transfer address goes to
> a DllMain, where certain DLL initialization is done, preparing the other
> entry points for use.

Right, modulo terminology: there's only one "entry point" in a PE format file.

> With an EXE, the transfer address goes to "main".

Sorry, no, the EXE entry point goes to a routine of no arguments.

Typically, in C and C++ that's a run time library routine (e.g. with Microsoft's
run time library mainCRTStartup or one of its cousins) which in turn calls the
C/C++ "main", while in Pascal it runs the main program, so on.

Note that the EXE entry point is probably still incorrectly documented as
requiring WinMain signature -- it's the most infamous Microsoft documentation
screw-up.

> So, when you load an EXE as a DLL, you will be RUNNING the program.

Sorry, no, that's not what happens.

The Windows API LoadLibrary(Ex) knows the difference between an EXE and a DLL.

It's documented as handling both sub-formats, and it does.

> That's is usually not what you want.

If that had happened then it would be a problem, yes. Happily it doesn't happen.
I've discussed the real problems else-thread.

Cheers & hth.,

- Alf

==============================================================================
TOPIC: sqlite3, memory db and multithreading
http://groups.google.com/group/comp.lang.python/t/fbe9426580697037?hl=en
==============================================================================

== 1 of 7 ==
Date: Fri, Mar 19 2010 12:20 am
From: Expo

On Mar 18, 1:58 pm, królewna <kr le...@ee.pl> wrote:
> The problem is simple: I have multiple threads within one program. At
> least 2 threads have to have access to in-memory sqlite database. It is
> not possible to pass sqlite objects to those threads because an
> exception is rised:
>
> ProgrammingError: SQLite objects created in a thread can only be used in
> that same thread.The object was created in thread id -1219066176 and
> this is thread id -1224475792
>
> Is there any EASY way to use this in-memory db in many threads? Creating
> another connection is not a solution as it creates completely new db
> instead of connecting to the existing one.
>

You can put the SQLite database into a Singleton class and use a
semaphore to serialize the access to methods which writes to the
database.

== 2 of 7 ==
Date: Fri, Mar 19 2010 3:55 am
From: krÃ³lewna

W dniu 18.03.2010 15:49, DreiJane pisze:

> Principally sqlite connections (sqlite3 objects in the C-API) can be
> used over multiple threads - and connections to :memory: make no
> difference. There are additional parameters to open() giving fine-
> tuned control. And apsw is promising a true reflection of sqlite's C-
> API.

It's my fault not saying what am I using to connect to sqlite db. It's
sqlite3 module. So there is no open() function etc. As for now I'm not
interested in rewriting my program to use apsw so is there any possible
way of working this out with sqlite3?

--
best regards
princess

== 3 of 7 ==
Date: Fri, Mar 19 2010 3:56 am
From: krÃ³lewna

W dniu 18.03.2010 23:06, Aahz pisze:
>
> You probably need to serialize access to the database through one thread.

sqlite3 objects are not pickable so it's not proper way.

--
best regards
princess

== 4 of 7 ==
Date: Fri, Mar 19 2010 4:04 am
From: krÃ³lewna

W dniu 19.03.2010 08:20, Expo pisze:
>
> You can put the SQLite database into a Singleton class and use a
> semaphore to serialize the access to methods which writes to the
> database.

I've tried this out but doesnt work. Still gives an error like:

ProgrammingError: SQLite objects created in a thread can only be used in
that same thread.The object was created in thread id -1216280896 and
this is thread id -1217107088

--
best regards
princess

== 5 of 7 ==
Date: Fri, Mar 19 2010 4:10 am
From: Tim Golden

On 19/03/2010 10:56, królewna wrote:
> W dniu 18.03.2010 23:06, Aahz pisze:
>>
>> You probably need to serialize access to the database through one thread.
>
> sqlite3 objects are not pickable so it's not proper way.

Is it possible you've misunderstood the meaning of the word "serialize"
here? What's being suggested isn't serialising (ie marshalling, pickling)
the data; rather, serialising the *access*, ie pushing all db requests into
a queue which is read by one thread which manages the only db connection.

TJG

== 6 of 7 ==
Date: Fri, Mar 19 2010 4:24 am
From: królewna

W dniu 19.03.2010 12:10, Tim Golden pisze:
> Is it possible you've misunderstood the meaning of the word "serialize"
> here?
It's not possible, it just happened :)

> What's being suggested isn't serialising (ie marshalling, pickling)
> the data; rather, serialising the *access*, ie pushing all db requests into
> a queue which is read by one thread which manages the only db connection.

That would make structure of program much more complicated. I would have
to create queue for putting there queries and some other extra
variables/structure to receive output from db and some more for
controlling the execution flow of awaiting threads.
--
best regards
princess

== 7 of 7 ==
Date: Fri, Mar 19 2010 6:58 am
From: krÃ³lewna

I have found the solution. I dont know why in python documentation there
is not a single word about this option. I've found it in pysqlite doc
site. So we have to add a new keyword argument to connection function
and we will be able to create cursors out of it in different thread. So use:

sqlite.connect(":memory:", check_same_thread = False)

works out perfectly for me. Of course from now on me need to take care
of safe multithreading access to the db. Anyway thx all for trying to help.

--
best regards
princess

==============================================================================
TOPIC: logging: local functions ==> loss of lineno
http://groups.google.com/group/comp.lang.python/t/1282b9ec08adc2a6?hl=en
==============================================================================

== 1 of 4 ==
Date: Fri, Mar 19 2010 12:56 am
From: Peter Otten <__peter__@web.de>

Hellmut Weber wrote:

> your hack is exactly what I was looking for.
> It permits to configure my logging messages as I want, e.g. using
> different colors for different classes of messages.
>
> I do not yet understand all details WHY it is working but suppose some
> study of the logging module will help me to understand.

Have a look at Logger.findCaller() in logging/__init__.py. To find the
calling function it does something like

for frame in walk_callstack():
if filename_of(frame) == _srcfile:
continue
return filename_lineno_and_functionname_of(frame)

The _srcfile is normally logging/__init__.py, and it is skipped because a
user typically is interested only in what happens outside the logging
package.

My hack under the hood changes

filename_of(frame) == _srcfile

from a string comparison to a

filename_of(frame) in set_of_files

containment test (see SrcFile.__eq__() posted above).

Peter

== 2 of 4 ==
Date: Fri, Mar 19 2010 3:39 am
From: Jean-Michel Pichavant

Hellmut Weber wrote:
> Am 11.03.2010 12:14, schrieb Peter Otten:
>> Hellmut Weber wrote:
>>
>>> Logging works very well giving the filename and line number of the
>>> point
>>> where it is called. As long as I use the loggers directly.
>>> BUT when I have to wrap the logger call in some other function, I
>>> always
>>> get file name and line number of the call of the logger inside the
>>> wrapping function.
>>>
>>> Is there a possibility to get this information in this situation too?
>>
>> The official way is probably to write a custom Logger class that
>> overrides
>> the findCaller() method.
>>
>> Below is a hack that monkey-patches the logging._srcfile attribute to
>> ignore
>> user-specified modules in the call stack:
>>
>> $ cat wrapper.py
>> import logging
>> import os
>> import sys
>>
>> logger = logging.getLogger()
>>
>> class SrcFile(object):
>> def __init__(self, exclude_files):
>> self.files = set(exclude_files)
>> def __eq__(self, other):
>> return other in self.files
>>
>> def fixname(filename):
>> if filename.lower().endswith((".pyc", ".pyo")):
>> filename = filename[:-4] + ".py"
>> return os.path.normcase(filename)
>>
>> if "--monkey" in sys.argv:
>> print "patching"
>> logging._srcfile = SrcFile([logging._srcfile, fixname(__file__)])
>>
>> def warn(*args, **kw):
>> logger.warn(*args, **kw)
>>
>> $ cat main.py
>> import logging
>> logging.basicConfig(format="%(filename)s<%(lineno)s>: %(message)s")
>> import wrapper
>> wrapper.warn("foo")
>> wrapper.warn("bar")
>> wrapper.warn("baz")
>>
>> $ python main.py
>> wrapper.py<23>: foo
>> wrapper.py<23>: bar
>> wrapper.py<23>: baz
>>
>> $ python main.py --monkey
>> patching
>> main.py<4>: foo
>> main.py<5>: bar
>> main.py<6>: baz
>>
>> $ python -V
>> Python 2.6.4
>>
>> Peter
>
> Hi Peter,
> your hack is exactly what I was looking for.
> It permits to configure my logging messages as I want, e.g. using
> different colors for different classes of messages.
>
> I do not yet understand all details WHY it is working but suppose some
> study of the logging module will help me to understand.
>
> Thank you very much
>
_scrFile is a private attribute of the logging module. Don't change it.

As you said yourself, 'The official way is probably to write a custom
Logger class that overrides
the findCaller() method'

== 3 of 4 ==
Date: Fri, Mar 19 2010 5:24 am
From: Peter Otten <__peter__@web.de>

Jean-Michel Pichavant wrote:

> Hellmut Weber wrote:
>> Am 11.03.2010 12:14, schrieb Peter Otten:
>>> Hellmut Weber wrote:
>>>
>>>> Logging works very well giving the filename and line number of the
>>>> point
>>>> where it is called. As long as I use the loggers directly.
>>>> BUT when I have to wrap the logger call in some other function, I
>>>> always
>>>> get file name and line number of the call of the logger inside the
>>>> wrapping function.
>>>>
>>>> Is there a possibility to get this information in this situation too?
>>>
>>> The official way is probably to write a custom Logger class that
>>> overrides
>>> the findCaller() method.
>>>
>>> Below is a hack that monkey-patches the logging._srcfile attribute to
>>> ignore
>>> user-specified modules in the call stack:
>>>
>>> $ cat wrapper.py
>>> import logging
>>> import os
>>> import sys
>>>
>>> logger = logging.getLogger()
>>>
>>> class SrcFile(object):
>>> def __init__(self, exclude_files):
>>> self.files = set(exclude_files)
>>> def __eq__(self, other):
>>> return other in self.files
>>>
>>> def fixname(filename):
>>> if filename.lower().endswith((".pyc", ".pyo")):
>>> filename = filename[:-4] + ".py"
>>> return os.path.normcase(filename)
>>>
>>> if "--monkey" in sys.argv:
>>> print "patching"
>>> logging._srcfile = SrcFile([logging._srcfile, fixname(__file__)])
>>>
>>> def warn(*args, **kw):
>>> logger.warn(*args, **kw)
>>>
>>> $ cat main.py
>>> import logging
>>> logging.basicConfig(format="%(filename)s<%(lineno)s>: %(message)s")
>>> import wrapper
>>> wrapper.warn("foo")
>>> wrapper.warn("bar")
>>> wrapper.warn("baz")
>>>
>>> $ python main.py
>>> wrapper.py<23>: foo
>>> wrapper.py<23>: bar
>>> wrapper.py<23>: baz
>>>
>>> $ python main.py --monkey
>>> patching
>>> main.py<4>: foo
>>> main.py<5>: bar
>>> main.py<6>: baz
>>>
>>> $ python -V
>>> Python 2.6.4
>>>
>>> Peter
>>
>> Hi Peter,
>> your hack is exactly what I was looking for.
>> It permits to configure my logging messages as I want, e.g. using
>> different colors for different classes of messages.
>>
>> I do not yet understand all details WHY it is working but suppose some
>> study of the logging module will help me to understand.
>>
>> Thank you very much
>>
> _scrFile is a private attribute of the logging module. Don't change it.
>
> As you said yourself, 'The official way is probably to write a custom
> Logger class that overrides
> the findCaller() method'

OK, I tried the this approach, too:

import logging
import os
import sys

from logging import currentframe

def fixname(filename):
if filename.lower().endswith((".pyc", ".pyo")):
filename = filename[:-4] + ".py"
return os.path.normcase(filename)

class MyLogger(logging.Logger):
exclude_files = set([logging._srcfile, fixname(__file__)])

def findCaller(self):
"""
Find the stack frame of the caller so that we can note the source
file name, line number and function name.
"""
f = currentframe()
#On some versions of IronPython, currentframe() returns None if
#IronPython isn't run with -X:Frames.
if f is not None:
f = f.f_back
rv = "(unknown file)", 0, "(unknown function)"
while hasattr(f, "f_code"):
co = f.f_code
filename = os.path.normcase(co.co_filename)
if filename in self.exclude_files:
f = f.f_back
continue
rv = (filename, f.f_lineno, co.co_name)
break
return rv

if "--custom-logger" in sys.argv:
print "setting custom logger"
logging.setLoggerClass(MyLogger)
logging.root = MyLogger("root", logging.WARNING)

logger = logging.getLogger()
def warn(*args, **kw):
logger.warn(*args, **kw)

I had to duplicate the original findCaller() method with only one line
changed. This means I have now some code duplication and I still have to
monitor future changes in the logging source.

In this case the "official way" seems to be more intrusive than the "hack".

Peter

== 4 of 4 ==
Date: Fri, Mar 19 2010 6:49 am
From: Jean-Michel Pichavant

Peter Otten wrote:
> Jean-Michel Pichavant wrote:
>
>
>> Hellmut Weber wrote:
>>
>>> Am 11.03.2010 12:14, schrieb Peter Otten:
>>>
>>>> Hellmut Weber wrote:
>>>>
>>>>
>>>>> Logging works very well giving the filename and line number of the
>>>>> point
>>>>> where it is called. As long as I use the loggers directly.
>>>>> BUT when I have to wrap the logger call in some other function, I
>>>>> always
>>>>> get file name and line number of the call of the logger inside the
>>>>> wrapping function.
>>>>>
>>>>> Is there a possibility to get this information in this situation too?
>>>>>
>>>> The official way is probably to write a custom Logger class that
>>>> overrides
>>>> the findCaller() method.
>>>>
>>>> Below is a hack that monkey-patches the logging._srcfile attribute to
>>>> ignore
>>>> user-specified modules in the call stack:
>>>>
>>>> $ cat wrapper.py
>>>> import logging
>>>> import os
>>>> import sys
>>>>
>>>> logger = logging.getLogger()
>>>>
>>>> class SrcFile(object):
>>>> def __init__(self, exclude_files):
>>>> self.files = set(exclude_files)
>>>> def __eq__(self, other):
>>>> return other in self.files
>>>>
>>>> def fixname(filename):
>>>> if filename.lower().endswith((".pyc", ".pyo")):
>>>> filename = filename[:-4] + ".py"
>>>> return os.path.normcase(filename)
>>>>
>>>> if "--monkey" in sys.argv:
>>>> print "patching"
>>>> logging._srcfile = SrcFile([logging._srcfile, fixname(__file__)])
>>>>
>>>> def warn(*args, **kw):
>>>> logger.warn(*args, **kw)
>>>>
>>>> $ cat main.py
>>>> import logging
>>>> logging.basicConfig(format="%(filename)s<%(lineno)s>: %(message)s")
>>>> import wrapper
>>>> wrapper.warn("foo")
>>>> wrapper.warn("bar")
>>>> wrapper.warn("baz")
>>>>
>>>> $ python main.py
>>>> wrapper.py<23>: foo
>>>> wrapper.py<23>: bar
>>>> wrapper.py<23>: baz
>>>>
>>>> $ python main.py --monkey
>>>> patching
>>>> main.py<4>: foo
>>>> main.py<5>: bar
>>>> main.py<6>: baz
>>>>
>>>> $ python -V
>>>> Python 2.6.4
>>>>
>>>> Peter
>>>>
>>> Hi Peter,
>>> your hack is exactly what I was looking for.
>>> It permits to configure my logging messages as I want, e.g. using
>>> different colors for different classes of messages.
>>>
>>> I do not yet understand all details WHY it is working but suppose some
>>> study of the logging module will help me to understand.
>>>
>>> Thank you very much
>>>
>>>
>> _scrFile is a private attribute of the logging module. Don't change it.
>>
>> As you said yourself, 'The official way is probably to write a custom
>> Logger class that overrides
>> the findCaller() method'
>>
>
> OK, I tried the this approach, too:
>
> import logging
> import os
> import sys
>
> from logging import currentframe
>
> def fixname(filename):
> if filename.lower().endswith((".pyc", ".pyo")):
> filename = filename[:-4] + ".py"
> return os.path.normcase(filename)
>
> class MyLogger(logging.Logger):
> exclude_files = set([logging._srcfile, fixname(__file__)])
>
> def findCaller(self):
> """
> Find the stack frame of the caller so that we can note the source
> file name, line number and function name.
> """
> f = currentframe()
> #On some versions of IronPython, currentframe() returns None if
> #IronPython isn't run with -X:Frames.
> if f is not None:
> f = f.f_back
> rv = "(unknown file)", 0, "(unknown function)"
> while hasattr(f, "f_code"):
> co = f.f_code
> filename = os.path.normcase(co.co_filename)
> if filename in self.exclude_files:
> f = f.f_back
> continue
> rv = (filename, f.f_lineno, co.co_name)
> break
> return rv
>
> if "--custom-logger" in sys.argv:
> print "setting custom logger"
> logging.setLoggerClass(MyLogger)
> logging.root = MyLogger("root", logging.WARNING)
>
> logger = logging.getLogger()
> def warn(*args, **kw):
> logger.warn(*args, **kw)
>
> I had to duplicate the original findCaller() method with only one line
> changed. This means I have now some code duplication and I still have to
> monitor future changes in the logging source.
>
> In this case the "official way" seems to be more intrusive than the "hack".
>
> Peter
You are still accessing the private attribute of the module logging.
My previous remark was misleading, in fact there's nothing you can do
about it.
_srcfile is not meant to be used elsewhere than in the logging module
itself.

However, I don't wanna sound like I'm rejecting this solution, 1st the
OP is satisified with it and since this solution is working, it is still
more helpful than anyone noticing that you've accessed a private
attribute (some will successfully argue that python allows to do so).

At the very begining of this thread I've provided a complete different
approach, instead of using the builtin 'filename' and 'lineno' field of
the logging, use custom fileds with the extra parameter.

It has also some drawbacks but could be a possible alternative.

Cheers,

in test.py:

import logging
import inspect

_logger = logging.getLogger(__name__)

class Foo:
def __init__(self):
self._logger = _logger

def info(self, msg):
# this is the method you don't want to appear in the logs,
instead the lineno and filename of this method caller
previousFrame = inspect.currentframe().f_back
self._logger.info(msg,
extra={'custom_lineno':previousFrame.f_lineno, 'custom_filename':
previousFrame.f_code.co_filename})

if __name__ == '__main__':
_logger.handlers=[]
_logger.addHandler(logging.StreamHandler())
_logger.handlers[-1].setFormatter(logging.Formatter('file
%(custom_filename)s line %(custom_lineno)d : %(message)s'))
_logger.setLevel(logging.DEBUG)
foo = Foo()
foo.info('a foo info')

In [3]: run test.py
file test.py line 20 : a foo info

==============================================================================
TOPIC: installing something to a virtualenv when it's already in site-packages
http://groups.google.com/group/comp.lang.python/t/3a1a09af746c14db?hl=en
==============================================================================

== 1 of 2 ==
Date: Fri, Mar 19 2010 1:05 am
From: nbv4

I have ipython installed via apt. I can go to the command line and
type 'ipython' and it will work. If I try to install ipython to a
virtualenv, I get this:

$ pip install -E env/ ipython
Requirement already satisfied: ipython in /usr/share/pyshared
Installing collected packages: ipython
Successfully installed ipython

I want ipython in both site-packages as well as in my virtualenv. This
is bad because when I activate the virtualenv, site-packages
disappears and ipython is not available. A work around is to uninstall
ipython from apt, install to the virtualenv, then reinstall in apt. Is
there a better way?

== 2 of 2 ==
Date: Fri, Mar 19 2010 1:26 am
From: Rolando Espinoza La Fuente

On Fri, Mar 19, 2010 at 4:05 AM, nbv4 <nbvfour@gmail.com> wrote:
> I have ipython installed via apt. I can go to the command line and
> type 'ipython' and it will work. If I try to install ipython to a
> virtualenv, I get this:
>
> $ pip install -E env/ ipython
> Requirement already satisfied: ipython in /usr/share/pyshared
> Installing collected packages: ipython
> Successfully installed ipython
>
> I want ipython in both site-packages as well as in my virtualenv. This
> is bad because when I activate the virtualenv, site-packages
> disappears and ipython is not available. A work around is to uninstall
> ipython from apt, install to the virtualenv, then reinstall in apt. Is
> there a better way?

I use -U (--upgrade) to force the installation within virtualenv. e.g:

$ pip install -E env/ -U ipython

Regards,

Rolando

==============================================================================
TOPIC: exp_continue
http://groups.google.com/group/comp.lang.python/t/e4eae2b2d4e0a10d?hl=en
==============================================================================

== 1 of 1 ==
Date: Fri, Mar 19 2010 2:03 am
From: Pankaj

I am using pexpect module in python

I would like to know how I can use exp_continue here.

what is the alternative of exp_continue of perl/tcl in pexpect.

==============================================================================
TOPIC: Why this compile error?
http://groups.google.com/group/comp.lang.python/t/88dcdc704d5d185f?hl=en
==============================================================================

== 1 of 3 ==
Date: Fri, Mar 19 2010 2:37 am
From: Jimbo

Hello

Can you help me figure out why I am getting this compile error with my
program. The error occurs right at the bottom of my code & I have
commented where it occurs.

The error is:
[QUOTE]Expected an indented block[/QUOTE]

[CODE]"""
*Stock Data Builder*
Algorithm:
- Search website for stock
- Get website HTML source code
- Search code for target stock data(price,dividends)
- Add data to text file
"""

import sys

# Functions
def getSource(URL, sourceBuffer):
""" Retrieve HTML source code from websitr URL &
save in sourceBuffer """

return sourceBuffer

def getData(targetData, dataBuffer):
""" Searches the string dataBuffer for the occurence of target
data & returns that whole line """

def writeToFile(textFile, dataBuffer):
""" Writes data in string dataBuffer to text file """

# CHANGE this to try except to catch errors
fileT = open(textFile,'a')
fileT.write(dataBuffer)
return True; # function succeeded

def writeToFile(textFile, dataList):
""" Writes data in List dataList to text file """

# CHANGE this to try except to catch errors
fileT = open(textFile,'a')

for element in dataList:
fileT.write(element)
return True; # function succeeded

# Main program loop
def main():
programEnd = False;

while (programEnd == False):
#ERROR HERE?? - Error="Expected an indented block"
main()[/CODE]

== 2 of 3 ==
Date: Fri, Mar 19 2010 3:04 am
From: Bruno Desthuilliers

Jimbo a écrit :
> Hello
>
> Can you help me figure out why I am getting this compile error with my
> program. The error occurs right at the bottom of my code & I have
> commented where it occurs.
>
> The error is:
> [QUOTE]Expected an indented block[/QUOTE]
>
> [CODE]
(snip)
> # Main program loop
> def main():
> programEnd = False;
>
> while (programEnd == False):
> #ERROR HERE?? - Error="Expected an indented block"

Indeed. You *need* a stamement in the while block. If you want a noop,
then use the 'pass' statement:

while programEnd == False:
pass

But note that as-is, this will go into an infinite loop.

== 3 of 3 ==
Date: Fri, Mar 19 2010 3:11 am
From: Ulrich Eckhardt

Jimbo wrote:
> Can you help me figure out why I am getting this compile error with my
> program. The error occurs right at the bottom of my code & I have
> commented where it occurs.
[...]
> def main():
> programEnd = False;
>
> while (programEnd == False):
> #ERROR HERE?? - Error="Expected an indented block"
> main()[/CODE]

Was the comment there before or not? In any case, I believe a comment
doesn't qualify as indented block, if you want to leave the block empty
use 'pass' instead. Also, make sure you don't mix tabs and spaces, which is
also a popular way to confuse Python.

BTW: "while" doesn't need brackets, that's a C-ism. ;)

Cheers!

Uli

--
Sator Laser GmbH
Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932

==============================================================================
TOPIC: should writing Unicode files be so slow
http://groups.google.com/group/comp.lang.python/t/f76baa1b73e8d17e?hl=en
==============================================================================

== 1 of 2 ==
Date: Fri, Mar 19 2010 5:08 am
From: djc

Ben Finney wrote:
> djc <slais-www@ucl.ac.uk> writes:
>
>> I have a simple program to read a text (.csv) file
>
> Could you please:
>
> * simplify it further: make a minimal version that demonstrates the
> difference you're seeing, without any extraneous stuff that doesn't
> appear to affect the result.
>
> * make it complete: the code you've shown doesn't do anything except
> define some functions.
>
> In other words: please reduce it to a complete, minimal example that we
> can run to see the same behaviour you're seeing.
>

It is the minimal example. The only thing omited is the opt.parse code that
calls _file_to_files(infile, outfile_prefix, column, sep):

--
David Clark, MSc, PhD. UCL Centre for Publishing
Gower Str London WCIE 6BT
What sort of web animal are you?
<https://www.bbc.co.uk/labuk/experiments/webbehaviour>

== 2 of 2 ==
Date: Fri, Mar 19 2010 5:56 am
From: Ben Finney

djc <slais-www@ucl.ac.uk> writes:

> Ben Finney wrote:
> > Could you please:
> >
> > * simplify it further: make a minimal version that demonstrates the
> > difference you're seeing, without any extraneous stuff that doesn't
> > appear to affect the result.
> >
> > * make it complete: the code you've shown doesn't do anything except
> > define some functions.
> >
> > In other words: please reduce it to a complete, minimal example that we
> > can run to see the same behaviour you're seeing.
>
> It is the minimal example. The only thing omited is the opt.parse code
> that calls _file_to_files(infile, outfile_prefix, column, sep):

What happens, then, when you make a smaller program that deals with only
one file?

What happens when you make a smaller program that only reads the file,
and doesn't write any? Or a different program that only writes a file,
and doesn't read any?

It's these sort of reductions that will help narrow down exactly what
the problem is. Do make sure that each example is also complete (i.e.
can be run as is by someone who uses only that code with no additions).

--
\ "To have the choice between proprietary software packages, is |
`\ being able to choose your master. Freedom means not having a |
_o__) master." —Richard M. Stallman, 2007-05-16 |
Ben Finney

==============================================================================
TOPIC: Importing modules
http://groups.google.com/group/comp.lang.python/t/6cd82adac66233b5?hl=en
==============================================================================

== 1 of 1 ==
Date: Fri, Mar 19 2010 5:25 am
From: Dave Angel

Peyman Askari wrote:
> I want to write a function which imports modules the first time, and reloads them afterwards, but I am running into problems with global variables and exec. I will include a full script, but let me elaborate first.
>
> Essentially what you need is
>
> def import_or_reload():
> """assume we want to load or reload sys"""
> if 'sys' in dir():
> reload(sys)
> else:
> import sys
>
> but this runs into the problem that sys is imported within the local scope of the function, so you insert a global statement
>
>
>
> def import_or_reload2():
>
> """Add 'global sys'"""
> global sys
>
> if 'sys' in dir():
>
> reload(sys)
>
> else:
>
> import sys
>
> 'sys' is still not in dir() as dir() pertains to the local scope of the function, but one can get around this by creating a local modules list and adding the imported modules to it
>
>
>
>
>
> def import_or_reload3():
>
>
> """Add 'global modules'"""
>
> global sys
> global modules
>
>
>
> if 'sys' in modules:
>
>
> reload(sys)
>
>
> else:
>
>
> import sys
> modules.append('sys')
>
> now lets add a parameter to the function signature, so any module name can be passed as an argument and loaded
>
>
>
>
>
> def import_or_reload4(module_name):
>
>
>
> """Add exec"""
>
>
> exec 'global %s'%module_name
>
> global modules
>
>
>
>
>
> if module_name in modules:
>
>
>
> exec 'reload(%s)'%module_name
>
>
>
> else:
>
>
>
> exec 'import %s'%module_name
>
> exec 'modules.append(\'%s\')'%module_name
>
> but this doesn't work as global does not cooperate with exec
> is there a __reload__('X') function like there is an __import__('X') function?
>
> Also is there a better way to import modules at run time?
>
> Cheers and here is the test script in case you can't access the attachment
>
> def a():
> global modules
> global sys
> import sys
>
> modules.append('sys')
>
> def b():
> global modules
> global sys
>
> reload(sys)
>
> def c(module_name):
> global modules
> exec 'global %s'%module_name
> exec 'import %s'%module_name
>
> modules.append(module_name)
>
> def test():
> global modules
> global sys
>
> #create the module list to contain all the modules
> modules=[]
>
> print 'originally dir() returns:'
> print dir()
>
> a()
> print 'function a() properly imports the following module:'
> print sys
> print 'is %s in %s->%s'%('sys',modules,'sys' in modules)
>
> b()
> print 'function b() properly reloads the following module:'
> print sys
> print 'is %s still in %s->%s'%('sys',modules,'sys' in modules)
>
> try:
> c('os')
> print 'function c() properly imports the following module:'
> except:
> print 'function c() failed to import module os'
> print 'is %s in %s->%s'%('os',modules,'os' in modules)
>
> try:
> print os
> print 'is %s still in %s->%s'%('os',modules,'os' in modules)
> except:
> print 'os was loaded, but is not visible outside of the scope of c()'
> --- On Fri, 3/19/10, python-list-request@python.org <python-list-request@python.org> wrote:
>
> From: python-list-request@python.org <python-list-request@python.org>
> Subject: Python-list Digest, Vol 78, Issue 192
> To: python-list@python.org
> Received: Friday, March 19, 2010, 7:05 AM
>
> <snip>
(When starting a new thread, create a new message addressed to
python-list@python.org, do not just reply to an existing message, (or
digest, which you did here). Some people actually try to follow
threads, and the software to do that uses more information than just the
message subject)

First comment. I would seriously try to avoid using reload() in
production code. The problems that can result are subtle. I use it for
debugging sessions, but not in real programs.

But I'll assume you have a use case (which would have been good to
explain), and have rejected the other possibilities.

Next, I'll point out that reloading sys isn't desirable, and it's one of
the specifically proscribed modules for reloading. But probably you
weren't really using sys, you were just sanitizing the code since you
knew we all had sys.

Next, if your reason for reloading is that you just changed the module
programmatically, and it might have been loaded by some other module (as
sys is, for example, long before your code starts), then you're not
checking in the right place. Instead of looking at your own global
space, you should be looking at sys.modules to decide whether something
has been loaded.

Perhaps the function you're looking for is imp.load_module()

DaveA

==============================================================================
TOPIC: C-API PyObject_Call
http://groups.google.com/group/comp.lang.python/t/03c70fd75579881f?hl=en
==============================================================================

== 1 of 2 ==
Date: Fri, Mar 19 2010 6:12 am
From: moerchendiser2k3

In my case I call a funcion and I would like to get the line
where the function returned.

for instance:

def my_function():
return 3

So I would like to get line 2(in C) somehow.

== 2 of 2 ==
Date: Fri, Mar 19 2010 7:11 am
From: Stefan Behnel

moerchendiser2k3, 19.03.2010 14:12:
> In my case I call a funcion and I would like to get the line
> where the function returned.
>
> for instance:
>
>
> def my_function():
> return 3
>
> So I would like to get line 2(in C) somehow.

Ok, so you're looking for the C-level trace function in CPython, I guess.

Could you explain why you want to know the line number?

Stefan

==============================================================================
TOPIC: What is pkg-config for ?
http://groups.google.com/group/comp.lang.python/t/30e184c7f1de5a62?hl=en
==============================================================================

== 1 of 1 ==
Date: Fri, Mar 19 2010 8:32 am
From: Johny

Can anyone explain what is pkg-config for?ANd how can I find it in
Windows?
Thanks

==============================================================================
TOPIC: Importing v reloading modules modules
http://groups.google.com/group/comp.lang.python/t/6e7db19869fb0f5f?hl=en
==============================================================================

== 1 of 1 ==
Date: Fri, Mar 19 2010 10:09 am
From: Peter Peyman Puk

Hello

I submitted this earlier today, but I was not clear enough, so I am
posting it again.

I am running a simulator written in python. The simulator has a small
TextView (actually a SourceView) widget which lets the user writes
scripts, and when they are satisfied they can execute that script to
get results. For arguments sake, we write a simple script and save it
as A.py and we import it and execute it more or less like so.

import A

#assume there is a function called test() in module A
A.test()

Then the user modifies the contents of A.py and saves it again (to
A.py) now all we have to do is the following

if 'A' in dir():
reload(A)
else:
import A

A.test()

But since the user chooses the file name, and not me, the programmer,
the module names will vary. Let's assume the module names are loaded
and stored in the list module_names, and we iterate over them, and
pass them as arguments to a function to import or reload each model as
appropriate

def import_or_reload(module_name):

if module_name in sys.modules:
#somehow reload
else:
#somehow import

does anyone know how to deal with the reload and import as they both
present problems since module_name is a string, and to reload
something along the lines of the below must be executed

exec 'reload(%s)'%module_name

and then we also have to deal with the scope issue since the loaded
module will be local and not global. I can execute something like so

exec 'global %s'%module_name

but that does not work very well with exec

any suggestions?

Cheers

Peyman

==============================================================================

You received this message because you are subscribed to the Google Groups "comp.lang.python"
group.

To post to this group, visit http://groups.google.com/group/comp.lang.python?hl=en

To unsubscribe from this group, send email to comp.lang.python+unsubscribe@googlegroups.com

To change the way you get mail from this group, visit:
http://groups.google.com/group/comp.lang.python/subscribe?hl=en

To report abuse, send email explaining the problem to abuse@googlegroups.com

==============================================================================
Google Groups: http://groups.google.com/?hl=en

twitter

Friday, March 19, 2010

comp.lang.python - 25 new messages in 11 topics - digest

0 Comments:

Post a Comment

About Me

Previous Posts