Reaching the limit of 500 local variables at startup
Reaching the limit of 500 local variables at startup
Posted on by Admin
5 thoughts on “Reaching the limit of 500 local variables at startup”
We’re running some code on the GL865-QUAD. We’ve apparently reached the limit of 500 local variables which results in a MemoryError when loading up modules. Before we refactor the code, I’d like to get some clarification on how the variable names are shared and counted.
If I have two functions in one module like this
def foo(response):
return response.split(",")
def bar(response):
return response.split()
Does the module calculate the response variable name once or twice during startup?
How about a class and names within the class:
class Foo:
def __init__(self, bar, baf):
self.bar = bar
self.baf = baf
def process(self, var):
self.bar = int(var)
def test(self, var):
self.baf = str(var)
Does the variable name var get counted once or twice?
Finally, how are dictionaries counted? Are dictionaries counted as one name, or is it possible that the keys and values count as additional names during startup?
Hi,
the module calculates the response variable name once
the module calculates the var variable name once
This request:
Are dictionaries counted as one name, or is it possible that the keys and values count as additional names during startup?
is not so clear. However if Iunderstand your question, keyscounts foronce
I enclose here a script useful to monitor the names: dict_debug.py (rename txt in py).
sys.getinterned() is an internal method which permits to list the names allocated in the dictionary. Using MemCheck method you could find how many locations are free (– whreaes the dictionary is not full yet : MemCheck() in the script generates strings to increase the dictionary until the exception occurrs). Then you need to comment the call of the method when you need to debug your original script.
Strategies to save space for the names list could be:
1- re-use a same variable name in different .py files 2-consider that no space is allocated in the names list for the strings delimited by ” or "" AND ending with r.
In this case you could try to use a method similar to the method removeCRfrom(string) to use strings and save space in the dictionary.:
def removeCRfrom(string):
a=string.endswith(‘r’) if a==1: b=string.split(‘r’) #print b[0] #print b[1] return b[0] else: print "the string doesn’t finish with r " return ”
Pay attention that ALL the strings with the same name in ALL py files of your project (e.g. ‘same_name’) have to be transformed in name +r (e.g.’same_namer’), otherwise you will not have any benefit
the scripts enclosed are "as it is" . they are not official scripts.
Another cause of the Memory Error could be some pyo files. As written in the Easy Script user guide, the recommended dimension of the compiled file .pyo should be <16KByte . This is a guard value. That means that some pyo files could have a size >=16KB without to cause the Memory Error
Hi,
the module calculates the response variable name once the module calculates the var variable name once
This request:
Are dictionaries counted as one name, or is it possible that the keys and values count as additional names during startup?
is not so clear. However if I understand your question, keys counts for once
I enclose here a script useful to monitor the names: dict_debug.py (rename txt in py). sys.getinterned() is an internal method which permits to list the names allocated in the dictionary.
Using MemCheck method you could find how many locations are free (– whreaes the dictionary is not full yet : MemCheck() in the script generates strings to increase the dictionary until the exception occurrs). Then you need to comment the call of the method when you need to debug your original script.
Strategies to save space for the names list could be:
1- re-use a same variable name in different .py files 2-consider that no space is allocated in the names list for the strings delimited by ” or "" AND ending with r.
In this case you could try to use a method similar to the method removeCRfrom(string) to use strings and save space in the dictionary:
def removeCRfrom(string):
a=string.endswith(‘r’) if a==1: b=string.split(‘r’) #print b[0] #print b[1] return b[0] else: print "the string doesn’t finish with r " return ”
Pay attention that ALL the strings with the same name in ALL py files of your project (e.g. ‘same_name’) have to be transformed in name +r (e.g.’same_namer’), otherwise you will not have any benefit
the scripts enclosed are "as it is" . they are not official scripts.
Another cause of the Memory Error could be some pyo files. As written in the Easy Script user guide, the recommended dimension of the compiled file .pyo should be <16KByte . This is a guard value. That means that some pyo files could have a size >=16KB without to cause the Memory Error
Thank you for the reply,
Regarding the second point. Did I understand correctly that every string that does not meet the criteria will consume a name?
As an example, I might have a function which wraps the SER.send() -function so that I can call it from multiple places and perhaps do some operations which are conditional of the input.
def debug(identifier, data):
…..
return SER.send("%s, %srn" % (identifier, data))
and I call it like this
debug("An identifier", "A debug message")
debug("Another id", "Another important debug message")
Do the two calls consume 2 names per call as none of the string arguments end with an ‘r’?
None of the compiled files are closing on the 16Kb limit so I doubt that is an issue here.
Hi,
you have all the code. I propose this: could you verify by yourself reusing dict_debug.py file that I have attached in my previous e-mail and then post the results?
We use cookies to enhance your browsing experience and help us improve our websites. To improve our website, we carefully select third parties that use cookies to allow us to serve specific content and achieve the purposes set out in our cookie policy. For more information on how to make adjustments through your browser to the cookies being used on your device, please click Find Out More link. By closing this banner or continuing to browse our website, you agree to our use of such cookies. FIND OUT MORE
We’re running some code on the GL865-QUAD. We’ve apparently reached the limit of 500 local variables which results in a MemoryError when loading up modules. Before we refactor the code, I’d like to get some clarification on how the variable names are shared and counted.
If I have two functions in one module like this
Does the module calculate the response variable name once or twice during startup?
How about a class and names within the class:
Does the variable name var get counted once or twice?
Finally, how are dictionaries counted? Are dictionaries counted as one name, or is it possible that the keys and values count as additional names during startup?
Hi,
the module calculates the response variable name once
the module calculates the var variable name once
This request:
is not so clear. However if I understand your question, keys counts for once
I enclose here a script useful to monitor the names: dict_debug.py (rename txt in py).
sys.getinterned() is an internal method which permits to list the names allocated in the
dictionary.
Using MemCheck method you could find how many locations are free (– whreaes the dictionary is not full yet : MemCheck() in the script generates strings to increase the dictionary until the exception occurrs). Then you need to comment the call of the method when you need to debug your original script.
Strategies to save space for the names list could be:
1- re-use a same variable name in different .py files
2-consider that no space is allocated in the names list for the strings delimited by ” or "" AND ending with r.
In this case you could try to use a method similar to the method
removeCRfrom(string) to use strings and save space in the dictionary.:
def removeCRfrom(string):
a=string.endswith(‘r’)
if a==1:
b=string.split(‘r’)
#print b[0]
#print b[1]
return b[0]
else:
print "the string doesn’t finish with r "
return ”
Pay attention that ALL the strings with the same name in ALL py files of your project (e.g.
‘same_name’) have to be transformed in name +r (e.g.’same_namer’), otherwise you will not have any benefit
the scripts enclosed are "as it is" . they are not official scripts.
Another cause of the Memory Error could be some pyo files. As written in the Easy Script user guide, the recommended dimension of the compiled file .pyo should be <16KByte . This is a guard value. That means that some pyo files could have a size >=16KB without to cause the Memory Error
Hi,
the module calculates the response variable name once
the module calculates the var variable name once
This request:
is not so clear. However if I understand your question, keys counts for once
I enclose here a script useful to monitor the names: dict_debug.py (rename txt in py).
sys.getinterned() is an internal method which permits to list the names allocated in the
dictionary.
Using MemCheck method you could find how many locations are free (– whreaes the dictionary is not full yet : MemCheck() in the script generates strings to increase the dictionary until the exception occurrs). Then you need to comment the call of the method when you need to debug your original script.
Strategies to save space for the names list could be:
1- re-use a same variable name in different .py files
2-consider that no space is allocated in the names list for the strings delimited by ” or "" AND ending with r.
In this case you could try to use a method similar to the method
removeCRfrom(string) to use strings and save space in the dictionary:
def removeCRfrom(string):
a=string.endswith(‘r’)
if a==1:
b=string.split(‘r’)
#print b[0]
#print b[1]
return b[0]
else:
print "the string doesn’t finish with r "
return ”
Pay attention that ALL the strings with the same name in ALL py files of your project (e.g.
‘same_name’) have to be transformed in name +r (e.g.’same_namer’), otherwise you will not have any benefit
the scripts enclosed are "as it is" . they are not official scripts.
Another cause of the Memory Error could be some pyo files. As written in the Easy Script user guide, the recommended dimension of the compiled file .pyo should be <16KByte . This is a guard value. That means that some pyo files could have a size >=16KB without to cause the Memory Error
Thank you for the reply,
Regarding the second point. Did I understand correctly that every string that does not meet the criteria will consume a name?
As an example, I might have a function which wraps the SER.send() -function so that I can call it from multiple places and perhaps do some operations which are conditional of the input.
and I call it like this
Do the two calls consume 2 names per call as none of the string arguments end with an ‘r’?
None of the compiled files are closing on the 16Kb limit so I doubt that is an issue here.
Hi,
you have all the code. I propose this: could you verify by yourself reusing dict_debug.py file that I have attached in my previous e-mail and then post the results?