Host compatibility testing

SDK for VST 3 audio plug-in and host development.
Post Reply
User avatar
lorcan
Posts: 20
Joined: Tue Jun 21, 2016 12:04 pm
Location: France
Contact:

Host compatibility testing

Post by lorcan » Thu Apr 13, 2017 12:08 pm

Hi all,

I'm currently testing my plug-ins in Cubase versions 8 and 9 under macOS and Windows to ensure everything as expected.
I also run the VST3 test host unit tests from time to time, as well the VSTScanner that was posted here, and Wavelab LE.
I assume Nuendo uses the same engine as Cubase under the hood, so testing with Cubase should be sufficient (except with surround plug-ins).

I'd like to know what Steinberg recommendations / best practices are in that regard, and possibly what kind of tests you run for your own plug-ins.
I have bought a license for Cubase + regular updates, and it really is not possible to purchase a Nuendo license for testing purposes.

On this topic, Avid have published a detailed test sheet for testing AAX plug-ins inside ProTools etc., and I think that's quite a neat idea.
Would you consider publishing some kind of recommended/required conformance, feature and stability test sheet for VST/VST3 ?

Thanks,
Lorcan
Lorcan Mc Donagh | lmdsp audio plug-ins

ygrabit
Posts: 150
Joined: Fri Jun 17, 2016 7:52 am
Location: Hamburg

Re: Host compatibility testing

Post by ygrabit » Tue Apr 18, 2017 6:39 am

Hi

good idea, we will provide later on a kind of guideline for testing inside a host like Cubase.
For now you can use the one provided by Avid...
Cheers
YVan Grabit

User avatar
lorcan
Posts: 20
Joined: Tue Jun 21, 2016 12:04 pm
Location: France
Contact:

Re: Host compatibility testing

Post by lorcan » Tue Apr 18, 2017 11:54 am

Hi Yvan, good to know thanks.
So in you opinion I can just skip testing Nuendo and I should be fine ?

Also I wanted to say the 'Unit test' feature in the VST3 Plugin Test Host is really great.
However I haven't found a way to launch the unit test for a specific plug-in from the command line,
which would allow using it inside an automated build/testing environment.
Perhaps this could be put on the idea/todo list ?

Cheers,
Lorcan
Lorcan Mc Donagh | lmdsp audio plug-ins

Arne Scheffler
Posts: 343
Joined: Mon Jun 20, 2016 7:53 am

Re: Host compatibility testing

Post by Arne Scheffler » Tue Apr 18, 2017 3:08 pm

lorcan wrote: Also I wanted to say the 'Unit test' feature in the VST3 Plugin Test Host is really great.
However I haven't found a way to launch the unit test for a specific plug-in from the command line,
which would allow using it inside an automated build/testing environment.
Perhaps this could be put on the idea/todo list ?
Use the vstvalidator (included in the sdk, see public.sdk/samples/vst/validator/. It's exactly what you want.

Cheers
Arne

User avatar
lorcan
Posts: 20
Joined: Tue Jun 21, 2016 12:04 pm
Location: France
Contact:

Re: Host compatibility testing

Post by lorcan » Tue Apr 18, 2017 3:29 pm

Arne, thanks, yes sorry I forgot about this one - I have built it and do use it too.

I'm just wondering if it performs the same checks as the UI version.
When I run vst3 validator (the one with sources in public.sdk/samples), it reports only 35 tests,
where as the binary UI version displays 'Number of test: 16815'

Cheers,
Lorcan
Lorcan Mc Donagh | lmdsp audio plug-ins

Arne Scheffler
Posts: 343
Joined: Mon Jun 20, 2016 7:53 am

Re: Host compatibility testing

Post by Arne Scheffler » Thu Apr 20, 2017 6:50 am

lorcan wrote:I'm just wondering if it performs the same checks as the UI version.
When I run vst3 validator (the one with sources in public.sdk/samples), it reports only 35 tests,
where as the binary UI version displays 'Number of test: 16815'
The VST3PluginTestHost checks more speaker arrangement combinations and has more parameter change tests than the command line tool. And a few tests that we haven't ported yet to the validator. We may do this in the future.

User avatar
lorcan
Posts: 20
Joined: Tue Jun 21, 2016 12:04 pm
Location: France
Contact:

Re: Host compatibility testing

Post by lorcan » Thu Apr 20, 2017 1:27 pm

Thanks for the clarification, Arne.

One thing I noticed is my plug-ins all pass all tests, except for one in the GUI version:
Global Instance > VST3 Conformity > Persistence > Parameter Bypass persistence
-> The Bypass Parameter is not in sync in the controller !
What is strange is that everything works as expected in the host, especially bypassing.
I've double checked my code and the ParameterInfo::kIsBypass flag is indeed present for the master bypass.
Lorcan Mc Donagh | lmdsp audio plug-ins

programaudio
Posts: 14
Joined: Mon Oct 03, 2016 4:09 am
Location: Chicago
Contact:

Re: Host compatibility testing

Post by programaudio » Sun May 03, 2020 12:10 am

How should I go about troubleshooting "the bypass parameter is not in sync with the controller!" error from the validator?
What scenarios would cause this?

The processor has implemented the
setState(IBStream*) method

and the EditController has implemented the
itilialize(FUnkown*),
setComponentState(IBStream*)
setParamNormalized(ParamID, ParamValue) methods

programaudio
Posts: 14
Joined: Mon Oct 03, 2016 4:09 am
Location: Chicago
Contact:

Re: Host compatibility testing

Post by programaudio » Sun May 03, 2020 2:08 pm

public.sdk\source\vst\testsuite\state\bypasspersistence provides "the bypass parameter is not in sync with the controller!" error message.

Found that the processor getState(IBStream*) was missing one of the parameters; issue resolved.

Post Reply