Repository navigation
refactor: public Keras APIs only, no input_shape warnings - #30
Merged
Merged
Conversation
…private copy ComplexConv imported conv_utils from tensorflow.python.keras.utils, a private, frozen copy of Keras 2 inside TensorFlow. Neither Keras 2 nor Keras 3 makes these helpers public, so complexnn/conv_utils.py now holds the five the layers use: normalize_tuple, normalize_padding, normalize_data_format, conv_output_length and deconv_output_length. They return the same results as TensorFlow's functions, checked against the private module on TF 2.15.1 and TF 2.21.0, with two differences. Invalid padding raises ValueError instead of failing an assert. normalize_data_format reads the public Keras backend: the private copy read its own legacy configuration, so ComplexConv2D and ComplexConv3D ignored tf.keras.backend.set_image_data_format(), and now they follow it. WeightNorm_Conv still subclasses the private Conv base. It is broken on both Keras versions and needs a rebuild on the public Layer class, so that import stays for now.
init.py imported serialize_keras_object and deserialize_keras_object from tensorflow.python.keras.utils.generic_utils, TensorFlow's private copy of Keras 2, and never used them. A new test parses every module in the package and fails on imports from tensorflow.python, keras.src, tf_keras.src or keras._tf_keras. The only allowed entry is the Conv base of WeightNorm_Conv, which stays until that class is rebuilt.
Drop imports that nothing uses: numpy in norm.py, Lambda in utils.py, tensorflow.keras.optimizers in fft.py and pool.py, and the fft functions in conv.py. conv.py replaced ifft and ifft2 with its own stubs straight after importing them, and never used fft or fft2. As a side effect, "import complexnn" no longer loads complexnn.fft, so code that reaches the module as an attribute needs "import complexnn.fft" first. The fft module was already broken on Keras 2 and Keras 3.
ComplexConv.call and WeightNorm_Conv.call set _keras_shape on their kernel tensors. The attribute comes from Keras 1; Keras 2 and Keras 3 never read it. One of the writes also added a tuple to a tensor, which TensorFlow turned into an elementwise sum instead of a shape. The channel axis and input_dim that ComplexConv.call computed only fed these writes, so they go as well. The outputs do not change. A new test fails if any module sets _keras_shape again.
ruff 0.17 joins implicit string concatenations that fit on one line. conv.py was the only module that "ruff format --check" rejected on main; the three error messages read the same as before.
Keras 3 warns "Do not pass an `input_shape`/`input_dim` argument to a layer" whenever Layer.__init__ receives either argument. ComplexDense, ComplexConv1D/2D/3D, ComplexBatchNormalization, ComplexLayerNorm and LayerNormalization forward their keyword arguments to Layer.__init__, so the README example and any code that passes input_dim raised this warning on Keras 3. Each of these layers now calls Layer.__init__ inside a small context manager in complexnn.utils that ignores only this message. Keras itself still records the shape, so Keras 3's Sequential still adds an input layer for it, and Keras 2, which never emits the warning, behaves as before. Other warnings raised during construction still show. The new tests check that constructing each layer with input_shape or input_dim raises no such warning, that Sequential still picks up the shape on both Keras versions, and that the helper hides nothing else.
_ignore_input_shape_warning entered warnings.catch_warnings around every Layer.__init__ call: on every Keras version, and whether or not the layer received input_shape or input_dim. catch_warnings resets the record Python keeps of warnings it has already shown. So a warning under the default "once per location" action showed again after each ComplexDense, ComplexConv, ComplexBN, ComplexLayerNorm or LayerNormalization was built, on Keras 2 and tf-keras as well, where the filter has nothing to hide. The helper now takes the layer's kwargs and enters catch_warnings only on Keras 3 (detected once at import through the public keras.version()) and only when kwargs holds input_shape or input_dim. In every other case the block does nothing. A regression test warns from one place three times, builds a layer after each warning, and checks that the warning is shown once. It fails on the previous commit in all three setups and passes on main. The helper tests now cover both paths: it hides only the input_shape warning on Keras 3, and filters nothing when the warning cannot fire.
The no-warning test counted only warnings containing "Do not pass an `input_shape`/`input_dim` argument", the same text the filter in complexnn/utils.py matches. If a Keras release rewords the message, the filter stops matching and the warning comes back, but the test stops matching it too and still passes. input_shape_warnings() now returns every warning that mentions input_shape or input_dim. It finds none on Keras 3, Keras 2 or tf-keras today. With Keras 3's warning reworded through a shim in keras.src.layers.layer, it reports the warning for all 8 cases, where the old check reported none. The exact text stays only in TestIgnoreInputShapeWarning, which checks the filter itself, with a comment saying the filter depends on Keras's wording.
… Keras WeightNorm_Conv still subclasses Conv from tensorflow.python.keras.layers.convolutional, the last private import in the package. conv.py imported it unguarded at module level and complexnn/__init__.py imports conv, so if a TensorFlow release drops that private copy of Keras 2, "import complexnn" fails, the failure this branch set out to remove. The import now sits in a try block after the package's own imports. If it raises ImportError, conv.py defines a stand-in Conv whose constructor raises ImportError with a message naming WeightNorm_Conv. Every other layer keeps working; only building a WeightNorm_Conv fails, and that class is already listed as not working in the README. When the private module is present, WeightNorm_Conv subclasses the same Conv as before. The new test blocks the module through a None entry in sys.modules, imports complexnn afresh, builds a ComplexConv2D, and checks that WeightNorm_Conv raises the ImportError. It fails on the previous commit in all three setups. The allow-list in test_public_api.py still names this one import.
complexnn/conv_utils.py, added earlier on this branch, is derived from TensorFlow's tensorflow/python/keras/utils/conv_utils.py: same functions, signatures, control flow and error messages, apart from the two changes its docstring lists. That code is under the Apache License 2.0, whose section 4(a) asks that recipients get a copy of the licence. Until now keras-complex only imported the TensorFlow module at runtime, so it redistributed nothing; LICENSE.md had only the two MIT notices. - conv_utils.py starts with TensorFlow's Apache header, copied from the original file, and a line saying keras-complex modified it. - LICENSE.md gets a third section naming the file, the TensorFlow copyright and the licence notice. It stays plain paragraphs, because docs/source/license.rst includes it as reStructuredText; docutils parses it without messages. - LICENSE-APACHE holds the full licence text, byte for byte the canonical file (md5 3b83ef96387f14655fc854ddc3c6bd57). setuptools' default licence-file patterns pick it up: a test build lists it in the wheel's dist-info and in the sdist, and METADATA gains "License-File: LICENSE-APACHE".
…rocess Importing complexnn a second time in the test process left Keras's registry of serialisable classes pointing at the throwaway copies, which breaks later tests that load models once the classes are registered.
…3 accepts Keras 3.3 rejects output_padding=0 in Conv1DTranspose, which made the suite fail on the oldest Keras 3 that tensorflow>=2.15 allows. Zero output padding is now checked against hand-computed lengths.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stage 2, PR 1 of 4. Moves complexnn onto public Keras APIs and stops the Keras 3
input_shapewarnings.What changes
conv_utilsfunctions the layers use come from TensorFlow's private copy of Keras 2. They now live incomplexnn/conv_utils.py, with TensorFlow's Apache 2.0 licence shipped next to it.init.pyand other unused imports are gone.import complexnnsurvives TensorFlow dropping its private Keras.WeightNorm_Conv, already broken and listed as such, is the one class still on the private base. Its import is guarded, so only building aWeightNorm_Convfails. A test checks this in a child process.input_shape/input_dim. The layers keep accepting both arguments. On Keras 3 that warning is filtered, and only while the layer is built. Trade-off, stated in the helper's docstring and recorded in the suite as an expected failure: on Keras 3, building a layer withinput_shape/input_dimresets which once-only warnings Python has shown._keras_shapewrites are removed.Behaviour change
ComplexConv2D/ComplexConv3Dwithdata_format=Nonenow followtf.keras.backend.set_image_data_format(). Before, they only readkeras.json. This matches built-in Keras layers, but a model that setchannels_firstglobally while feedingchannels_lastdata to these layers will now see a different layout.Verification
test_conv_utils: 13 passed, 264 subtestsruff format --check, pre-commit and the flake8 critical check are clean.Merge notes
Merge order for the four Stage 2 PRs: this one, then serialisation, then validation and docs, then tooling. The later ones need small rebases, mostly import unions. The validation and docs PR's layout page says the convolutions read
keras.json; after this PR they follow the runtime setting, and that sentence changes in its rebase.