Dark Mode

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Recipe for argon2-cffi#2398

Merged
AndreMiras merged 18 commits intokivy:developfrom
Arjun-Somvanshi:develop
Jan 5, 2021
Merged

Recipe for argon2-cffi#2398
AndreMiras merged 18 commits intokivy:developfrom
Arjun-Somvanshi:develop

Conversation

Copy link
Contributor

Arjun-Somvanshi commented Jan 3, 2021

What have I done so far?

In my code I have been using the pycryptodome lib which already has a recipe in p4a, since it seemed to have the same dependencies as argon2-cffi, that being: cffi and setuptools, I decided to use it's code for argon2's recipe. After I did the fork I added the dependency in my spec file. It is visible that it got downloaded (but probably is not getting built properly) but always fails when I run the app. This is the error I got on a logcat after the build was done:

`working: Traceback (most recent call last): [0m
working: File "setup.py", line 13, in [0m
working: from setuptools import find_packages, setup [0m
working: ModuleNotFoundError: No module named 'setuptools' [0m

Command failed: /usr/bin/python3 -m pythonforandroid.toolchain create --dist_name=reminiscor --bootstrap=sdl2 --requirements=python3,kivy,pycryptodome,argon2-cffi --arch armeabi-v7a --copy-libs --color=always --storage-dir="/home/kivy/Reminiscor/.buildozer/android/plat form/build-armeabi-v7a" --ndk-api=21

Copy link
Member

inclement commented Jan 3, 2021

Try using a CompiledComponentsPythonRecipe - you'll need to do something like this to get the argon C components to build.

Arjun-Somvanshi reacted with thumbs up emoji

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 3, 2021

I tried to look into what is a CompiledComponentsPythonRecipe but this is what it says in the documentation:

Using a CompiledComponentsPythonRecipe

This is similar to a CythonRecipe but is intended for modules like numpy which include compiled but non-cython components. It uses a similar mechanism to compile with the right environment.

This isn't documented yet because it will probably be changed so that CythonRecipe inherits from it (to avoid code duplication).

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 3, 2021

Is this link relevant to what you think I should look for : (https://askpythonquestions.com/2020/12/13/kivy-python-for-android-how-to-create-custom-recipes/)

Copy link
Contributor

obfusk commented Jan 3, 2021 *
edited
Loading

You may (also) need to set call_hostpython_via_targetpython = False; e.g.

from pythonforandroid.recipe import CompiledComponentsPythonRecipe


class Argon2Recipe(CompiledComponentsPythonRecipe):
version = '20.1.0'
url = 'https://github.com/hynek/argon2-cffi/archive/master.zip'
depends = ['setuptools', 'cffi']
call_hostpython_via_targetpython = False


recipe = Argon2Recipe()
Arjun-Somvanshi reacted with thumbs up emoji

obfusk reviewed Jan 3, 2021
url = 'https://github.com/hynek/argon2-cffi/archive/master.zip'
depends = ['setuptools', 'cffi']
call_hostpython_via_targetpython = False

Copy link
Contributor

obfusk Jan 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

flake8 requires an extra blank line here.

Copy link
Contributor Author

Arjun-Somvanshi Jan 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay

Copy link
Contributor

obfusk Jan 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it doesn't like the fact that the "blank line" contains whitespace.

AndreMiras reacted with thumbs up emoji
Copy link
Contributor Author

Arjun-Somvanshi commented Jan 3, 2021

From what I understand it's still not able to compile the c?

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 3, 2021

From what I understand it's still not able to compile the c?

It can't find the argon.h file do I need to include the c code somehow?

Copy link
Contributor

obfusk commented Jan 3, 2021

It can't find the argon.h file do I need to include the c code somehow?

Looks like it uses a git submodule for the extras/libargon2 directory, which is not included in the .zip.

Copy link
Contributor

obfusk commented Jan 3, 2021 *
edited
Loading

Try using url = 'git+https://github.com/hynek/argon2-cffi@{version}'. I think that will include submodules.

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 3, 2021

It can't find the argon.h file do I need to include the c code somehow?

Looks like it uses a git submodule for the extras/libargon2 directory, which is not included in the .zip.

shall I make my own zip with the other repo included inside this one from: https://github.com/p-h-c/phc-winner-argon2/tree/62358ba2123abd17fccf2a108a301d4b52c01a7c

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 3, 2021

Try using url = 'git+https://github.com/hynek/argon2-cffi@{version}'.

okay, sorry I typed my thing before seeing your comment

Copy link
Contributor

obfusk commented Jan 3, 2021 *
edited
Loading

Try using url = 'git+https://github.com/hynek/argon2-cffi@{version}'. I think that will include submodules.

Actually, I think that should just be url = 'git+https://github.com/hynek/argon2-cffi'.

Normally, you should include {version} in the url, but the @{version} syntax for git urls is for pip, whereas p4a seems to use the recipe's .version instead (and check out that tag/branch).

obfusk reviewed Jan 3, 2021
class Argon2Recipe(CompiledComponentsPythonRecipe):
version = '20.1.0'
url = 'https://github.com/hynek/argon2-cffi/archive/master.zip'
url = 'url = 'git+https://github.com/hynek/argon2-cffi@{version}'
Copy link
Contributor

obfusk Jan 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that doesn't look quite right :)

Copy link
Contributor Author

Arjun-Somvanshi Jan 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is my first pr ever, sorry if it's a bit shabby.

Copy link
Contributor

obfusk Jan 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

np. though you may want to do a rebase and squash all those intermediate commits when it's working.

Copy link
Contributor Author

Arjun-Somvanshi Jan 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah okay I will do that

obfusk reviewed Jan 3, 2021

class Argon2Recipe(CompiledComponentsPythonRecipe):
version = '20.1.0'
url = 'git+https://github.com/hynek/argon2-cffi@{version}'
Copy link
Contributor

obfusk Jan 3, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to remove the @{version} here. Sorry.

Copy link
Contributor Author

Arjun-Somvanshi Jan 3, 2021 *
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

np. I didn't see you edited that message.

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 3, 2021 *
edited
Loading

Okay, I don't quite understand what the issue is now? It seemed like it almost passed it this time though

Copy link
Contributor

obfusk commented Jan 3, 2021

I think the problem is that p4a calls setup_ext but not setup_clib.
Try adding build_cmd = 'build' to the recipe. That should call both.

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 4, 2021

You may (also) want to try building the recipe on your own computer instead.
With buildozer, you can use e.g.

p4a.branch = develop
p4a.local_recipes = ./my-p4a-recipes

I don't understand how the p4a.local_recipes works, this is what I have so far:

# (str) python-for-android fork to use, defaults to upstream (kivy)
p4a.fork = Arjun-Somvanshi

# (str) python-for-android branch to use, defaults to master
p4a.branch = develop

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =

If I am using my own fork, then why does it need the local_recipes thing?

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 4, 2021 *
edited
Loading

Incase you are wondering if I have done a build with those settings above in me .spec file, here is the output
build_output.txt

Copy link
Contributor

obfusk commented Jan 4, 2021

If I am using my own fork, then why does it need the local_recipes thing?

In that case it doesn't :)
I'm using it so that I don't need to use a fork (and keep it up to date) just to have an extra recipe.

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 4, 2021 *
edited
Loading

Incase you are wondering if I have done a build with those settings above in me .spec file, here is the output
build_output.txt

Oh okay, so if I use a local recipe do I still need to specify the package in the requirement field? Also what do you think of that build output I uploaded, is that some mistake on my part?

Copy link
Contributor

obfusk commented Jan 4, 2021

This recipe seems to work:

from pythonforandroid.recipe import CompiledComponentsPythonRecipe


class Argon2Recipe(CompiledComponentsPythonRecipe):
version = '20.1.0'
url = 'git+https://github.com/hynek/argon2-cffi'
depends = ['setuptools', 'cffi']
call_hostpython_via_targetpython = False
build_cmd = 'build'

def get_recipe_env(self, arch):
env = super().get_recipe_env(arch)
env['ARGON2_CFFI_USE_SSE2'] = '0'
return env


recipe = Argon2Recipe()

Copy link
Contributor

obfusk commented Jan 4, 2021

You may run into #2400 when testing that locally.

AndreMiras previously approved these changes Jan 5, 2021
Copy link
Member

AndreMiras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR and the follow up changes with the help of @inclement and @obfusk
Looking good, will merge after the CI builds green

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 5, 2021

i'll do a rebase and squash the commits

Arjun-Somvanshi dismissed AndreMiras's stale review via 5b37ba7 January 5, 2021 06:02
Copy link
Contributor Author

Arjun-Somvanshi commented Jan 5, 2021

I just added the change in the url

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 5, 2021

yeah, so as I thought, the libargon2 is missing hence we used that url from before, so this time I am gonna make the changes, squash the commits, and force push.

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 5, 2021

I am getting a lot of merge conflicts even though I followed the steps to a git rebase properly

Copy link
Member

AndreMiras commented Jan 5, 2021

Thanks for giving it a try
Weird for the rebase. I can't think of what would have made conflicts, but that's fine we'll try to deal without.
FYI what I would have tried is:

git fetch --all --prune
# squashing
git reset --soft HEAD~17
git commit --amend
# rebasing
git rebase upstream/develop # given the remote is called upstream

I'll try to at least squash from the GitHub interface at least because it allows to

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 5, 2021 *
edited
Loading

Thanks for giving it a try
Weird for the rebase. I can't think of what would have made conflicts, but that's fine we'll try to deal without.
FYI what I would have tried is:

git fetch --all --prune
# squashing
git reset --soft HEAD~17
git commit --amend
# rebasing
git rebase upstream/develop # given the remote is called upstream

I'll try to at least squash from the GitHub interface at least because it allows to
I'm sorry, I was trying to post the comment and closed the pr by mistake, the issue I think is that I mixed up the commits, I did a few directly from github instead of the command line

Arjun-Somvanshi closed this Jan 5, 2021
Arjun-Somvanshi reopened this Jan 5, 2021
Copy link
Contributor Author

Arjun-Somvanshi commented Jan 5, 2021 *
edited
Loading

okay tests passed, no more commits now. Just gotta squash the commits from before.

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 5, 2021

@AndreMiras I tried the git commands you suggested the git log on my local repo gives the 17 commits even after running those commands

Copy link
Member

AndreMiras commented Jan 5, 2021

No problem, thanks again for trying

AndreMiras approved these changes Jan 5, 2021
Copy link
Member

AndreMiras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good thanks!

AndreMiras merged commit aadcfc4 into kivy:develop Jan 5, 2021
Copy link
Contributor

obfusk commented Jan 5, 2021

git fetch --all --prune
# squashing
git reset --soft HEAD~17
git commit --amend
# rebasing
git rebase upstream/develop # given the remote is called upstream

Using an interactive rebase to squash always works for me:

git fetch --all
git rebase -i upstream/develop
AndreMiras reacted with heart emoji

Copy link
Contributor

obfusk commented Jan 5, 2021

yeah, so as I thought, the libargon2 is missing hence we used that url from before, so this time I am gonna make the changes, squash the commits, and force push.

An alternative solution would be to add a separate recipe for libargon2 and depend on that.
This might be a good idea anyway, since that can then be updated separately (and shared by other recipes).

AndreMiras reacted with heart emoji

Copy link
Contributor Author

Arjun-Somvanshi commented Jan 5, 2021

@obfusk just give me a few days, I am in the middle of my semester exams, I will get back to this and clean it up as well as try and do this the right way. :) Thanks a lot for your help.

Copy link
Contributor

obfusk commented Jan 5, 2021

just give me a few days, I am in the middle of my semester exams

np. There's no hurry :) Good luck!

I will get back to this and clean it up

The current version is squashed and merged. So it's already fine :)

as well as try and do this the right way.

There's nothing "wrong" now. Using a separate libargon2 recipe would just be a nice improvement IMHO. And should be a separate PR.

Arjun-Somvanshi reacted with thumbs up emoji

vesellov pushed a commit to vesellov/python-for-android that referenced this pull request Feb 10, 2021
* testing argon2-cffi
* Using CompiledComponentsPythonRecipe
* argon2-cffi recipe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

AndreMiras AndreMiras approved these changes

+1 more reviewer

obfusk obfusk left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants