Skip to content

Commit 854cfa1

Browse files
authored
Merge pull request #11 from objectbox/small-scalars
Scalar types support
2 parents dd2dde1 + b0de1f1 commit 854cfa1

File tree

10 files changed

+124
-55
lines changed

10 files changed

+124
-55
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# IDE
22
.idea
3+
.vscode
34

45
# Environment
56
.venv/

CONTRIBUTING.md

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
11
Contributing
22
------------------
3-
Anyone can contribute, be it by coding, improving docs or just proposing a new feature.
3+
Anyone can contribute, be it by coding, improving docs or just proposing a new feature.
44
As a new contributor, you may want to have a look at some of the following issues:
5-
* [**good first issue**](https://github.com/objectbox/objectbox-python/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) tag
5+
* [**good first issue**](https://github.com/objectbox/objectbox-python/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) tag
66
* [**help wanted**](https://github.com/objectbox/objectbox-python/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) tag
77

8-
When picking up an existing issue, please let others know in the issue comment.
8+
When picking up an existing issue, please let others know in the issue comment.
99
Don't hesitate to reach out for guidance or to discuss a solution proposal!
1010

1111
### Code contributions
@@ -16,20 +16,19 @@ When creating a Pull Request for code changes, please check that you cover the f
1616
### Basic technical approach
1717
ObjectBox offers a [C API](https://github.com/objectbox/objectbox-c) which can be integrated into python using
1818
[ctypes](https://docs.python.org/dev/library/ctypes.html).
19-
The C API is is also used by the ObjectBox language bindings for [Go](https://github.com/objectbox/objectbox-go),
19+
The C API is is also used by the ObjectBox language bindings for [Go](https://github.com/objectbox/objectbox-go),
2020
[Swift](https://github.com/objectbox/objectbox-swift), and [Dart/Flutter](https://github.com/objectbox/objectbox-dart).
2121
These language bindings currently serve as an example for this Python implementation.
2222
Internally, ObjectBox uses [FlatBuffers](https://google.github.io/flatbuffers/) to store objects.
2323

24-
The main prerequisite to using the Python APIs is the ObjectBox binary library (.so, .dylib, .dll depending on your
25-
platform) which actually implements the database functionality. The library should be placed in the
26-
`objectbox/lib/[architecture]/` folder of the checked out repository. You can get/update it by running `make get-lib`.
24+
The main prerequisite to using the Python APIs is the ObjectBox binary library (.so, .dylib, .dll depending on your
25+
platform) which actually implements the database functionality. The library should be placed in the
26+
`objectbox/lib/[architecture]/` folder of the checked out repository. You can get/update it by running `make depend`.
2727

2828
### Getting started as a contributor
2929
#### Initial setup
3030
If you're just getting started, run the following simple steps to set up the repository on your machine
3131
* clone this repository
32-
* `pip install virtualenv` install [virtualenv](https://pypi.org/project/virtualenv/) if you don't have it yet
3332
* `make depend` to initialize `virtualenv` and get dependencies (objectbox-c shared library)
3433
* `make` to build and test
3534

Makefile

Lines changed: 23 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
1+
SHELL := /bin/bash
12
VENV = .venv
23
VENVBIN = ${VENV}/bin
4+
PYTHON = python3
5+
PIP = ${PYTHON} -m pip
36

47
# Detect windows - works on both 32 & 64-bit windows
58
ifeq ($(OS),Windows_NT)
@@ -9,7 +12,7 @@ endif
912
export PATH := $(abspath ${VENVBIN}):${PATH}
1013

1114

12-
.PHONY: init test build benchmark publish
15+
.PHONY: init test build benchmark publish venv-init
1316

1417
# Default target executed when no arguments are given to make.
1518
default_target: build test
@@ -22,33 +25,43 @@ help: ## Show this help
2225
all: depend build test ## Get dependencies, clean, build and test
2326

2427
build: ${VENV} clean ## Clean and build
25-
python setup.py bdist_wheel
28+
set -e ; \
29+
${PYTHON} setup.py bdist_wheel ; \
2630
ls -lh dist
2731

2832
${VENV}: ${VENVBIN}/activate
2933

30-
${VENVBIN}/activate: requirements.txt
31-
virtualenv ${VENV}
34+
venv-init:
35+
${PIP} install --user virtualenv
36+
${PYTHON} -m virtualenv ${VENV}
37+
3238
# remove packages not in the requirements.txt
33-
pip3 freeze | grep -v -f requirements.txt - | grep -v '^#' | grep -v '^-e ' | xargs pip3 uninstall -y || echo "never mind"
3439
# install and upgrade based on the requirements.txt
35-
python -m pip install --upgrade -r requirements.txt
3640
# let make know this is the last time requirements changed
41+
${VENVBIN}/activate: requirements.txt
42+
set -e ; \
43+
if [ ! -d "${VENV}" ] ; then make venv-init ; fi ; \
44+
${PIP} freeze | grep -v -f requirements.txt - | grep -v '^#' | grep -v '^-e ' | xargs ${PIP} uninstall -y || echo "never mind" ; \
45+
${PIP} install --upgrade -r requirements.txt ; \
3746
touch ${VENVBIN}/activate
3847

3948
depend: ${VENV} ## Prepare dependencies
40-
python download-c-lib.py
49+
set -e ; \
50+
${PYTHON} download-c-lib.py
4151

4252
test: ${VENV} ## Test all targets
43-
python -m pytest --capture=no --verbose
53+
set -e ; \
54+
${PYTHON} -m pytest --capture=no --verbose
4455

4556
benchmark: ${VENV} ## Run CRUD benchmarks
46-
python -m benchmark
57+
set -e ; \
58+
${PYTHON} -m benchmark
4759

4860
clean: ## Clean build artifacts
4961
rm -rf build/
5062
rm -rf dist/
5163
rm -rf *.egg-info
5264

5365
publish: ## Publish the package built by `make build`
54-
python -m twine upload --verbose dist/objectbox*.whl
66+
set -e ; \
67+
${PYTHON} -m twine upload --verbose dist/objectbox*.whl

README.md

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -41,8 +41,19 @@ from objectbox.model import *
4141
@Entity(id=1, uid=1)
4242
class Person:
4343
id = Id(id=1, uid=1001)
44-
first_name = Property(str, id=2, uid=1002)
45-
last_name = Property(str, id=3, uid=1003)
44+
name = Property(str, id=2, uid=1002)
45+
is_enabled = Property(bool, id=3, uid=1003)
46+
# int can be stored with 64 (default), 32, 16 or 8 bit precision.
47+
int64 = Property(int, id=4, uid=1004)
48+
int32 = Property(int, type=PropertyType.int, id=5, uid=1005)
49+
int16 = Property(int, type=PropertyType.short, id=6, uid=1006)
50+
int8 = Property(int, type=PropertyType.byte, id=7, uid=1007)
51+
# float can be stored with 64 or 32 (default) bit precision.
52+
float64 = Property(float, id=8, uid=1008)
53+
float32 = Property(float, type=PropertyType.float, id=9, uid=1009)
54+
byte_array = Property(bytes, id=10, uid=1010)
55+
# Regular properties are not stored.
56+
transient = ""
4657
```
4758

4859
### Using ObjectBox
@@ -58,16 +69,16 @@ import objectbox
5869

5970
# Configure ObjectBox: should be done only once in the whole program and the "ob" variable should be kept around
6071
model = objectbox.Model()
61-
model.entity(Person, last_property_id=objectbox.model.IdUid(3, 1003))
72+
model.entity(Person, last_property_id=objectbox.model.IdUid(10, 1010))
6273
model.last_entity_id = objectbox.model.IdUid(1, 1)
6374
ob = objectbox.Builder().model(model).directory("db").build()
6475

6576
# Open the box of "Person" entity. This can be called many times but you can also pass the variable around
6677
box = objectbox.Box(ob, Person)
6778

68-
id = box.put(Person(first_name="Joe", last_name="Green")) # Create
79+
id = box.put(Person(name="Joe Green")) # Create
6980
person = box.get(id) # Read
70-
person.last_name = "Black"
81+
person.name = "Joe Black"
7182
box.put(person) # Update
7283
box.remove(person) # Delete
7384
```

objectbox/model/__init__.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -22,5 +22,6 @@
2222
'Entity',
2323
'Id',
2424
'IdUid',
25-
'Property'
25+
'Property',
26+
'PropertyType'
2627
]

objectbox/model/entity.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,8 @@ def fill_properties(self):
6868
"programming error - invalid type OB & FB type combination"
6969
self.offset_properties.append(prop)
7070

71+
# print('Property {}.{}: {} (ob:{} fb:{})'.format(self.name, prop._name, prop._py_type, prop._ob_type, prop._fb_type))
72+
7173
if not self.id_property:
7274
raise Exception("ID property is not defined")
7375
elif self.id_property._ob_type != OBXPropertyType_Long:

objectbox/model/properties.py

Lines changed: 44 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -12,22 +12,54 @@
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
1414

15+
from enum import IntEnum
1516

1617
from objectbox.c import *
1718
import flatbuffers.number_types
1819

1920

20-
# base property
21+
class PropertyType(IntEnum):
22+
bool = OBXPropertyType_Bool
23+
byte = OBXPropertyType_Byte
24+
short = OBXPropertyType_Short
25+
char = OBXPropertyType_Char
26+
int = OBXPropertyType_Int
27+
long = OBXPropertyType_Long
28+
float = OBXPropertyType_Float
29+
double = OBXPropertyType_Double
30+
string = OBXPropertyType_String
31+
# date = OBXPropertyType_Date
32+
# relation = OBXPropertyType_Relation
33+
byteVector = OBXPropertyType_ByteVector
34+
# stringVector = OBXPropertyType_StringVector
35+
36+
37+
fb_type_map = {
38+
PropertyType.bool: flatbuffers.number_types.BoolFlags,
39+
PropertyType.byte: flatbuffers.number_types.Int8Flags,
40+
PropertyType.short: flatbuffers.number_types.Int16Flags,
41+
PropertyType.char: flatbuffers.number_types.Int8Flags,
42+
PropertyType.int: flatbuffers.number_types.Int32Flags,
43+
PropertyType.long: flatbuffers.number_types.Int64Flags,
44+
PropertyType.float: flatbuffers.number_types.Float32Flags,
45+
PropertyType.double: flatbuffers.number_types.Float64Flags,
46+
PropertyType.string: flatbuffers.number_types.UOffsetTFlags,
47+
# PropertyType.date: flatbuffers.number_types.Int64Flags,
48+
# PropertyType.relation: flatbuffers.number_types.Int64Flags,
49+
PropertyType.byteVector: flatbuffers.number_types.UOffsetTFlags,
50+
# PropertyType.stringVector: flatbuffers.number_types.UOffsetTFlags,
51+
}
52+
53+
2154
class Property:
22-
def __init__(self, py_type: type, id: int, uid: int):
55+
def __init__(self, py_type: type, id: int, uid: int, type: PropertyType = None):
2356
self._id = id
2457
self._uid = uid
25-
self._name = "" # set in Entity.fillProperties()
58+
self._name = "" # set in Entity.fill_properties()
2659

27-
self._fb_type = None # flatbuffers.number_types
2860
self._py_type = py_type
29-
self._ob_type = OBXPropertyType(0)
30-
self.__set_basic_type()
61+
self._ob_type = type if type != None else self.__determine_ob_type()
62+
self._fb_type = fb_type_map[self._ob_type]
3163

3264
self._is_id = isinstance(self, Id)
3365
self._flags = OBXPropertyFlags(0)
@@ -37,23 +69,18 @@ def __init__(self, py_type: type, id: int, uid: int):
3769
self._fb_slot = self._id - 1
3870
self._fb_v_offset = 4 + 2*self._fb_slot
3971

40-
def __set_basic_type(self) -> OBXPropertyType:
72+
def __determine_ob_type(self) -> OBXPropertyType:
4173
ts = self._py_type
4274
if ts == str:
43-
self._ob_type = OBXPropertyType_String
44-
self._fb_type = flatbuffers.number_types.UOffsetTFlags
75+
return OBXPropertyType_String
4576
elif ts == int:
46-
self._ob_type = OBXPropertyType_Long
47-
self._fb_type = flatbuffers.number_types.Int64Flags
77+
return OBXPropertyType_Long
4878
elif ts == bytes: # or ts == bytearray: might require further tests on read objects due to mutability
49-
self._ob_type = OBXPropertyType_ByteVector
50-
self._fb_type = flatbuffers.number_types.UOffsetTFlags
79+
return OBXPropertyType_ByteVector
5180
elif ts == float:
52-
self._ob_type = OBXPropertyType_Double
53-
self._fb_type = flatbuffers.number_types.Float64Flags
81+
return OBXPropertyType_Double
5482
elif ts == bool:
55-
self._ob_type = OBXPropertyType_Bool
56-
self._fb_type = flatbuffers.number_types.BoolFlags
83+
return OBXPropertyType_Bool
5784
else:
5885
raise Exception("unknown property type %s" % ts)
5986

tests/common.py

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -23,20 +23,26 @@ def autocleanup():
2323
def load_empty_test_objectbox(name: str = "") -> objectbox.ObjectBox:
2424
model = objectbox.Model()
2525
from objectbox.model import IdUid
26-
model.entity(TestEntity, last_property_id=IdUid(6, 1006))
26+
model.entity(TestEntity, last_property_id=IdUid(10, 1010))
2727
model.last_entity_id = IdUid(1, 1)
2828

2929
db_name = test_dir if len(name) == 0 else test_dir + "/" + name
3030

3131
return objectbox.Builder().model(model).directory(db_name).build()
3232

3333

34-
def assert_equal(actual, expected):
34+
def assert_equal_prop(actual, expected, default):
35+
assert actual == expected or (isinstance(
36+
expected, objectbox.model.Property) and actual == default)
37+
38+
39+
def assert_equal(actual: TestEntity, expected: TestEntity):
3540
"""Check that two TestEntity objects have the same property data"""
3641
assert actual.id == expected.id
37-
assert isinstance(expected.bool, objectbox.model.Property) or actual.bool == expected.bool
38-
assert isinstance(expected.int, objectbox.model.Property) or actual.int == expected.int
39-
assert isinstance(expected.str, objectbox.model.Property) or actual.str == expected.str
40-
assert isinstance(expected.float, objectbox.model.Property) or actual.float == expected.float
41-
assert isinstance(expected.bytes, objectbox.model.Property) or actual.bytes == expected.bytes
42-
42+
assert_equal_prop(actual.int64, expected.int64, 0)
43+
assert_equal_prop(actual.int32, expected.int32, 0)
44+
assert_equal_prop(actual.int16, expected.int16, 0)
45+
assert_equal_prop(actual.int8, expected.int8, 0)
46+
assert_equal_prop(actual.float64, expected.float64, 0)
47+
assert_equal_prop(actual.float32, expected.float32, 0)
48+
assert_equal_prop(actual.bytes, expected.bytes, b'')

tests/model.py

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,13 @@ class TestEntity:
66
id = Id(id=1, uid=1001)
77
str = Property(str, id=2, uid=1002)
88
bool = Property(bool, id=3, uid=1003)
9-
int = Property(int, id=4, uid=1004)
10-
float = Property(float, id=5, uid=1005)
11-
bytes = Property(bytes, id=6, uid=1006)
9+
int64 = Property(int, type=PropertyType.long, id=4, uid=1004)
10+
int32 = Property(int, type=PropertyType.int, id=5, uid=1005)
11+
int16 = Property(int, type=PropertyType.short, id=6, uid=1006)
12+
int8 = Property(int, type=PropertyType.byte, id=7, uid=1007)
13+
float64 = Property(float, type=PropertyType.double, id=8, uid=1008)
14+
float32 = Property(float, type=PropertyType.float, id=9, uid=1009)
15+
bytes = Property(bytes, id=10, uid=1010)
1216
transient = "" # not "Property" so it's not stored
1317

1418
def __init__(self, string: str = ""):

tests/test_box.py

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,9 +21,13 @@ def test_box_basics():
2121
object = TestEntity()
2222
object.id = 5
2323
object.bool = True
24-
object.int = 42
24+
object.int64 = 9223372036854775807
25+
object.int32 = 2147483647
26+
object.int16 = 32767
27+
object.int8 = 127
2528
object.str = "foo"
26-
object.float = 4.2
29+
object.float64 = 4.2
30+
object.float32 = 1.5
2731
object.bytes = bytes([1, 1, 2, 3, 5])
2832
object.transient = "abcd"
2933

@@ -67,7 +71,8 @@ def test_box_bulk():
6771

6872
box.put(TestEntity("first"))
6973

70-
objects = [TestEntity("second"), TestEntity("third"), TestEntity("fourth"), box.get(1)]
74+
objects = [TestEntity("second"), TestEntity("third"),
75+
TestEntity("fourth"), box.get(1)]
7176
box.put(objects)
7277
assert box.count() == 4
7378
assert objects[0].id == 2

0 commit comments

Comments
 (0)