Python - Brute Force - Memory Error

Asked

Viewed 133 times

2

I am working on this code that generates sequences with all the possibilities of the characters entered(your_list)

your_list = 'abcdefghijklmnopqrstuvwxyz'
complete_list = []
for current in range(10):
    a = [i for i in your_list]
    for y in range(current):
         a = [x+i for i in your_list for x in a]
    complete_list = complete_list+a
print (complete_list)

I’m finding the error highlighted below, but I don’t know how to solve this problem

"C:\Users\admin\PycharmProjects\markestrat\venv\Scripts\python.exe" "C:/Users/admin/PycharmProjects/markestrat/venv/brutef.py"
Traceback (most recent call last):
  File "C:/Users/admin/PycharmProjects/markestrat/venv/brutef.py", line 65, in <module>
    a = [x+i for i in your_list for x in a]
  File "C:/Users/admin/PycharmProjects/markestrat/venv/brutef.py", line 65, in <listcomp>
    a = [x+i for i in your_list for x in a]
MemoryError

Does anyone know how to solve this problem?

  • What would be the intended result? All combinations between letters of the alphabet? What sizes? Replacement or not? Does order matter? That is to say, ab and ba would be equivalent or not?

  • [ab] and [ba] would not be equivalent, for now I’m just testing, so the size does not make much difference now, a 4 or 5 characters is already good, but the problem is when I do something with much more characters, it accuses Memory Error, as in this example of 10 characters, but with 4 or 5, the program works normally and quickly, from the sixth character it accuses error

  • Yes, because you keep all combinations in a list. This is not necessary and already ahead for you to study generators.

1 answer

4


You can get the lowercase letters of the alphabet with:

from string import ascii_lowercase

If you want all letters, including uppercase, string.ascii_letters, or only capital letters, string.ascii_uppercase, or any available value in string.

To generate all possible combinations of N characters, without repetition, keeping the order of the same, you can use itertools.combinations:

from itertools import combinations
from string import ascii_lowercase

print(list(combinations(ascii_lowercase, 2)))

The return of this would be

[('a', 'b'), ('a', 'c'), ('a', 'd'), ..., ('x', 'y'), ('x', 'z'), ('y', 'z')]

Thus, you could define a function that generates all possible combinations without doing repetition:

def all_combinations(iterable):
    size = len(iterable)
    for i in range(1, size+1):
        yield from combinations(iterable, i)

This will generate all possible combinations of 1 to 26 characters, without repetition and maintaining order, but without storing all the values in memory. How generators are used, each value is generated on demand, implementing what we call Lazy-Evaluation and thus circumventing the memory problem presented in the question.

Notes

  1. The return is an eternal object of tuples with the combination; if it is desirable a string, such conversion should be treated;
  2. All combinations respect the order of the input characters, so the combination is generated ('a', 'b') but not the ('b', 'a');
  3. All combinations are without repetition, not generating sequences with the same character as for example ('a', 'a', 'a'); if desired with repetition, use itertools.combinations_with_replacement;
  • But how do I not give MEMORY ERROR? When the character size grows it Uga, in your code when I put 6-character combinations of the same error as mine (MEMORY ERROR), I wanted to be able to solve this problem and then add uppercase letters, number and special characters, and yes, I would need it to take repetitions into account and also present combinations such as ('a', 'b') and ('b', 'a')

  • @Luisfernandogarcia The only thing I focused on was precisely solving the memory problem by using generators instead of lists. If you kept giving the error, you probably tried to store everything in a list again. As for the case of considering repetitions or the order I do not include in the answer because I judged it to be outside the context of your question and put comments so that you could implement on your own.

  • I just increased the size of your "n", didn’t change anything -- print(list(Combinations(ascii_lowercase, 2))) -- put in place this 2 a 6

  • @Luisfernandogarcia The solution is the function I created, this line was only to exemplify the use/return of the function.

  • It worked here, thank you very much

Browser other questions tagged

You are not signed in. Login or sign up in order to post.