Distutils causes protobuf to fail
I have a simple .proto as follows:
syntax = "proto2";
package p2p;
message dataFrame {
repeated int64 row_label = 1;
repeated int64 col_label = 2;
message data_frame_row {
repeated int32 column = 1;
}
repeated data_frame_row row = 3;
}
which I compile with protoc.
I can compile and use the following (simplified) cpp code (transitMatrix.h #includes "p2p.pb.h"):
#include "transitMatrix.h"
int main()
{
transitMatrix matrix(5, true);
matrix.addEdgeToGraph(0, 1, 5, true);
matrix.addEdgeToGraph(1, 2, 6, true);
matrix.writeTMX("outfile.tmx");
return 0;
}
However, the transitMatrix class is intended to be cythonized with distutils.
In my (simplified) test.pyx file, I cdef extern both the transitMatrix class and the dataFrame class:
cdef extern from "src/utils/serializer/p2p.pb.cc" namespace "p2p":
cdef cppclass dataFrame:
dataFrame()
cdef extern from "src/transitMatrix.h":
cdef cppclass transitMatrix:
transitMatrix()
And in my setup.py, the distutils Extension is as follows:
EXTENSION = distutils.extension.Extension(
name = 'test', language = 'c++',
sources = ['test.pyx'],
extra_compile_args = ['-Wno-unused-function',
'-std=c++11', '-Wall', '-O3', '-lprotobuf'
] + ouff_mac,
undef_macros = ["NDEBUG"],
extra_link_args = ouff_mac
)
When I run python3 setup.py install, it succeeds but warns:
clang: warning: -lprotobuf: 'linker' input unused
Finally, when I try to import the generated class in iPython (transitMatrixAdapter), I get the following error:
ImportError: dlopen(/Users/user/anaconda3/lib/python3.6/site- packages/transitMatrixAdapter.cpython-36m-darwin.so, 2): Symbol not found: __ZN6google8protobuf7Message20DiscardUnknownFieldsEv
Referenced from: /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
Expected in: flat namespace in /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
I should add that this works as expected when the protobuf dependency is removed. Why is the .so not found? I presume this is a linker error, but where is the issue?
c++ python-3.x protocol-buffers cython distutils
asked Nov 20 at 19:42
lmnoel
62
add a comment |
I have a simple .proto as follows:
syntax = "proto2";
package p2p;
message dataFrame {
repeated int64 row_label = 1;
repeated int64 col_label = 2;
message data_frame_row {
repeated int32 column = 1;
}
repeated data_frame_row row = 3;
}
which I compile with protoc.
I can compile and use the following (simplified) cpp code (transitMatrix.h #includes "p2p.pb.h"):
#include "transitMatrix.h"
int main()
{
transitMatrix matrix(5, true);
matrix.addEdgeToGraph(0, 1, 5, true);
matrix.addEdgeToGraph(1, 2, 6, true);
matrix.writeTMX("outfile.tmx");
return 0;
}
However, the transitMatrix class is intended to be cythonized with distutils.
In my (simplified) test.pyx file, I cdef extern both the transitMatrix class and the dataFrame class:
cdef extern from "src/utils/serializer/p2p.pb.cc" namespace "p2p":
cdef cppclass dataFrame:
dataFrame()
cdef extern from "src/transitMatrix.h":
cdef cppclass transitMatrix:
transitMatrix()
And in my setup.py, the distutils Extension is as follows:
EXTENSION = distutils.extension.Extension(
name = 'test', language = 'c++',
sources = ['test.pyx'],
extra_compile_args = ['-Wno-unused-function',
'-std=c++11', '-Wall', '-O3', '-lprotobuf'
] + ouff_mac,
undef_macros = ["NDEBUG"],
extra_link_args = ouff_mac
)
When I run python3 setup.py install, it succeeds but warns:
clang: warning: -lprotobuf: 'linker' input unused
Finally, when I try to import the generated class in iPython (transitMatrixAdapter), I get the following error:
ImportError: dlopen(/Users/user/anaconda3/lib/python3.6/site- packages/transitMatrixAdapter.cpython-36m-darwin.so, 2): Symbol not found: __ZN6google8protobuf7Message20DiscardUnknownFieldsEv
Referenced from: /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
Expected in: flat namespace in /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
I should add that this works as expected when the protobuf dependency is removed. Why is the .so not found? I presume this is a linker error, but where is the issue?
c++ python-3.x protocol-buffers cython distutils
asked Nov 20 at 19:42
lmnoel
62
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 at 23:38
add a comment |
I have a simple .proto as follows:
syntax = "proto2";
package p2p;
message dataFrame {
repeated int64 row_label = 1;
repeated int64 col_label = 2;
message data_frame_row {
repeated int32 column = 1;
}
repeated data_frame_row row = 3;
}
which I compile with protoc.
I can compile and use the following (simplified) cpp code (transitMatrix.h #includes "p2p.pb.h"):
#include "transitMatrix.h"
int main()
{
transitMatrix matrix(5, true);
matrix.addEdgeToGraph(0, 1, 5, true);
matrix.addEdgeToGraph(1, 2, 6, true);
matrix.writeTMX("outfile.tmx");
return 0;
}
However, the transitMatrix class is intended to be cythonized with distutils.
In my (simplified) test.pyx file, I cdef extern both the transitMatrix class and the dataFrame class:
cdef extern from "src/utils/serializer/p2p.pb.cc" namespace "p2p":
cdef cppclass dataFrame:
dataFrame()
cdef extern from "src/transitMatrix.h":
cdef cppclass transitMatrix:
transitMatrix()
And in my setup.py, the distutils Extension is as follows:
EXTENSION = distutils.extension.Extension(
name = 'test', language = 'c++',
sources = ['test.pyx'],
extra_compile_args = ['-Wno-unused-function',
'-std=c++11', '-Wall', '-O3', '-lprotobuf'
] + ouff_mac,
undef_macros = ["NDEBUG"],
extra_link_args = ouff_mac
)
When I run python3 setup.py install, it succeeds but warns:
clang: warning: -lprotobuf: 'linker' input unused
Finally, when I try to import the generated class in iPython (transitMatrixAdapter), I get the following error:
ImportError: dlopen(/Users/user/anaconda3/lib/python3.6/site- packages/transitMatrixAdapter.cpython-36m-darwin.so, 2): Symbol not found: __ZN6google8protobuf7Message20DiscardUnknownFieldsEv
Referenced from: /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
Expected in: flat namespace in /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
I should add that this works as expected when the protobuf dependency is removed. Why is the .so not found? I presume this is a linker error, but where is the issue?
c++ python-3.x protocol-buffers cython distutils
asked Nov 20 at 19:42
lmnoel
62
I have a simple .proto as follows:
syntax = "proto2";
package p2p;
message dataFrame {
repeated int64 row_label = 1;
repeated int64 col_label = 2;
message data_frame_row {
repeated int32 column = 1;
}
repeated data_frame_row row = 3;
}
which I compile with protoc.
I can compile and use the following (simplified) cpp code (transitMatrix.h #includes "p2p.pb.h"):
#include "transitMatrix.h"
int main()
{
transitMatrix matrix(5, true);
matrix.addEdgeToGraph(0, 1, 5, true);
matrix.addEdgeToGraph(1, 2, 6, true);
matrix.writeTMX("outfile.tmx");
return 0;
}
However, the transitMatrix class is intended to be cythonized with distutils.
In my (simplified) test.pyx file, I cdef extern both the transitMatrix class and the dataFrame class:
cdef extern from "src/utils/serializer/p2p.pb.cc" namespace "p2p":
cdef cppclass dataFrame:
dataFrame()
cdef extern from "src/transitMatrix.h":
cdef cppclass transitMatrix:
transitMatrix()
And in my setup.py, the distutils Extension is as follows:
EXTENSION = distutils.extension.Extension(
name = 'test', language = 'c++',
sources = ['test.pyx'],
extra_compile_args = ['-Wno-unused-function',
'-std=c++11', '-Wall', '-O3', '-lprotobuf'
] + ouff_mac,
undef_macros = ["NDEBUG"],
extra_link_args = ouff_mac
)
When I run python3 setup.py install, it succeeds but warns:
clang: warning: -lprotobuf: 'linker' input unused
Finally, when I try to import the generated class in iPython (transitMatrixAdapter), I get the following error:
ImportError: dlopen(/Users/user/anaconda3/lib/python3.6/site- packages/transitMatrixAdapter.cpython-36m-darwin.so, 2): Symbol not found: __ZN6google8protobuf7Message20DiscardUnknownFieldsEv
Referenced from: /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
Expected in: flat namespace in /Users/user/anaconda3/lib/python3.6/site-packages/transitMatrixAdapter.cpython-36m-darwin.so
I should add that this works as expected when the protobuf dependency is removed. Why is the .so not found? I presume this is a linker error, but where is the issue?
c++ python-3.x protocol-buffers cython distutils
c++ python-3.x protocol-buffers cython distutils
asked Nov 20 at 19:42
lmnoel
62
asked Nov 20 at 19:42
lmnoel
62
asked Nov 20 at 19:42
lmnoel
62
asked Nov 20 at 19:42
lmnoel
62
asked Nov 20 at 19:42
lmnoel
62
62
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 at 23:38
add a comment |
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 at 23:38
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 at 20:35
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 at 23:38
Solved! Thank you very much
– lmnoel
Nov 20 at 23:38
add a comment |
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400414%2fdistutils-causes-protobuf-to-fail%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
active
oldest
votes
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400414%2fdistutils-causes-protobuf-to-fail%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
-lprotobuf should be in extra link args, as clang tries to warn you
– ead
Nov 20 at 20:35
Solved! Thank you very much
– lmnoel
Nov 20 at 23:38