Support

Site Restoration

#41349 ErrNo #1071 Specified key was too long; max key length is 1000 bytes

Posted in ‘Site restoration’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

PHP version
8.1
CMS Type
Joomla!
CMS Version
4.49
Backup Tool Version
Akeeba 9.9.10
Kickstart version
8.0

Latest post by nicholas on Monday, 25 November 2024 01:34 CST

KenHorse

I'm moving a Joomla 4.4.9 site from a development to production server. On the production server, I totally deleted everything from the webserver root, deleted the database and re-created it. Of course, the user account has full privileges.

Upon restoration, the database restore fails with the error in the title.

I tried the suggestions in the following link but they didn't work

https://www.akeeba.com/support/site-restoration/26830:admin-tools-table-in-dbse-causes-error-in-restoration.html

 

Our production site is down of course.

nicholas
Akeeba Staff
Manager

The table referenced in the error message has misconfigured indexes. They are not compatible with the utf8mb4 encoding.

You can of course restore without using the utf8mb4 encoding by selecting (checking) the “Force UTF-8 collation on tables” option and deselecting (remove the checkmark from) the “Allow UTF8MB4 auto-detection” option in the Database Restoration step of the restoration script.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

KenHorse

Could the version of MySQL have something to do with it? If I restore the exact same backup to my local development server (running 10.11.6-MariaDB) versus the production server (which throws the error and is running MySQL 5.7.23-23), it restores completely without error

 

 

KenHorse

I did try your suggestion and am still dealing with the same error. Site is still down too of course

 

Here is the complete error report:

 

CREATE TABLE `jos_menu` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `menutype` varchar(24) NOT NULL COMMENT 'The type of menu this item belongs to. FK to `#__menu_types`.menutype',
  `title` varchar(255) NOT NULL COMMENT 'The display title of the menu item.',
  `alias` varchar(400) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL COMMENT 'The SEF alias of the menu item.',
  `note` varchar(255) NOT NULL DEFAULT '',
  `path` varchar(1024) NOT NULL COMMENT 'The computed path of the menu item based on the alias field.',
  `link` varchar(1024) NOT NULL COMMENT 'The actually link the menu item refers to.',
  `type` varchar(16) NOT NULL COMMENT 'The type of link: Component, URL, Alias, Separator',
  `published` tinyint(4) NOT NULL DEFAULT 0 COMMENT 'The published state of the menu link.',
  `parent_id` int(10) unsigned NOT NULL DEFAULT 1 COMMENT 'The parent menu item in the menu tree.',
  `level` int(10) unsigned NOT NULL DEFAULT 0 COMMENT 'The relative level in the tree.',
  `component_id` int(10) unsigned NOT NULL DEFAULT 0 COMMENT 'FK to `#__extensions`.id',
  `checked_out` int(10) unsigned DEFAULT NULL,
  `checked_out_time` datetime DEFAULT NULL,
  `browserNav` tinyint(4) NOT NULL DEFAULT 0 COMMENT 'The click behaviour of the link.',
  `access` int(10) unsigned NOT NULL DEFAULT 0 COMMENT 'The access level required to view the menu item.',
  `img` varchar(255) NOT NULL COMMENT 'The image of the menu item.',
  `template_style_id` int(10) unsigned NOT NULL DEFAULT 0,
  `params` mediumtext NOT NULL COMMENT 'JSON encoded data for the menu item.',
  `lft` int(11) NOT NULL DEFAULT 0 COMMENT 'Nested set lft.',
  `rgt` int(11) NOT NULL DEFAULT 0 COMMENT 'Nested set rgt.',
  `home` tinyint(3) unsigned NOT NULL DEFAULT 0 COMMENT 'Indicates if this menu item is the home or default page.',
  `language` char(7) NOT NULL DEFAULT '',
  `client_id` tinyint(4) NOT NULL DEFAULT 0,
  `publish_up` datetime DEFAULT NULL,
  `publish_down` datetime DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `idx_client_id_parent_id_alias_language` (`client_id`,`parent_id`,`alias`,`language`) 
) ENGINE=MyISAM AUTO_INCREMENT=716 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci

nicholas
Akeeba Staff
Manager

Yes, the problem is the database version and the database server's configuration.

The root cause of this issue is that in Joomla! 4.4 the maintainers increased the alias column length from 191 to 400 characters. This means that if you put it in an index it requires 1600 bytes instead of 764 bytes in UTF8MB4 which uses 4 bytes per character (1200 instead of 573 in plain old UTF8 which uses 3 bytes per character). This is a problem because MySQL 5.7 has a default hard limit of 767 bytes for the InnoDB engine, and 1000 bytes for the MyISAM engine.

This change was thoughtlessly backported from Joomla! 5 which requires MySQL 8 or later. In MyQSL 8 the hard limit is higher for InnoDB, 3072 bytes if the DYNAMIC row format is used, this row format being the default for MySQL 8 anyway.

This could all have been avoided if the unique key definition had `alias`(100) instead of `alias`. That's database management 101, but I guess the Joomla! maintainers skipped class the day we were talking about index lengths. Well. You've hit a bug in Joomla! which won't be fixed because Joomla! 4.4 is in security maintenance mode and this is a functional, not security, issue.

As a side note, I can also tell that your MySQL 5.7 server does not support InnoDB which is actually required for Joomla! 4 and later. I can tell that because the restoration script noticed that and tried to use the MyISAM storage engine instead.

That said, there are things you –or, rather, the server administrator– can do.

If you can upgrade the server to MySQL 8 that would be the best idea, since it ensures you can upgrade to Joomla! 5 and beyond. Do note that MySQL 5.7 is already end of life since October 2023. So, upgrading is the sanest option from every perspective: future-proofing, performance, and security.

If upgrading to MySQL 8 is not an option at this time, you can ask the server administrator to enable the InnoDB engine on the MySQL 5.7server, and enable the innodb_large_prefix option. This will work around this problem without causing any functional issues on your site, and without compromising its long term stability. The fact remains that MySQL 5.7 is End of Life and needs to be upgraded ASAP, but at least this workaround gives you a way to have a working site in the meantime.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

KenHorse

Thanks, I will see if I can my get hosting service (HostMonster) to enable InnoDB and that option but they have been very firm that they will not update MySQL server (I am looking into changing hosts for this very reason). Interestingly, that website has been running 4.4.0 for over a year without any real issues overall. Only since trying to upgrade to 4.4.9 has this become a problem.

I'm going to try to restore the 4.4.0 site for now. Again, thanks for all your help with this.

nicholas
Akeeba Staff
Manager

Yes, as I said the change to that table only took place in 4.4.9. In previous versions of Joomla! 4 (from Joomla! 4.0 alpha 1 up to and including 4.4.8) the alias columns were defined as VARCHAR(191) for the index length reasons I mentioned. I was one of the developers involved in the original conversion of Joomla! 3 to UTF8MB4, this was a discussion we had back in the day. The change took place because one person had a problem with an alias that was longer than that.

I personally think it's having an alias even as long as 191 characters is ludicrous. Beyond around 70 characters of URL path length it's no longer a human-readable URL, and definitely not one a human can remember accurately. If you're not making URLs memorable to humans you might just as well use a random 32-character hex string as an alias, giving you over 2 billion possible articles within the same category with a much faster to index key. In other words, I find this change ridiculous and counter-productive. But I'm no longer involved in Joomla! maintenance in any capacity, so my opinion is just that.

As for your host, making it clear that they will stick with a database version which no longer receives security updates since October 2023 is beyond stupid. It's insane, in the literal sense. No sane person would choose to expose their clients and their infrastructure to security threats. They have their priorities wrong. You should indeed find a new host a.s.a.p. At this point it's more than an inconvenience, it's a security time bomb!

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

KenHorse

So you're saying that 4.4.1 does not have the database issue? Hmmmm...I am still receiving the same error when I try to restore the site via kickstart

nicholas
Akeeba Staff
Manager

Ah, you are right, the change was actually there in 4.4.1: https://github.com/joomla/joomla-cms/blob/4ee38101e132f2b422a9cd15a2cb69c24bca45bb/installation/sql/mysql/base.sql#L461 I have gotten the timeline wrong. It looks like the change on that table was made on 4.0. Which basically means that Joomla 4.4's claim to support MySQL 5.6 or later is dubious at best.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

KenHorse

Sorry to keep bothering you but it seems our site was able to be updated to Joomla 4.4.9 and everything seems to be working with one exception now. When I try to do an Akeeba backup (9.9.10), it fails with

"Field 'id' doesn't have a default value".

 

Is this fixable?

 

Honestly, we'd be willing to pay a reasonable fee to get this site running properly

nicholas
Akeeba Staff
Manager

Hm, this error message can be pretty ominous. It means the autonumber field is no longer autonumber. This requires doing something manually and explicitly to mess it up.

I will ask two questions.

  1. Are you able to create a new article? Even an unpublished one, just for testing, would do.
  2. How did you transfer your database content? Was it using CSV or XML files with phpMyAdmin?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

KenHorse

1) I tried to create a new article - "Save failed with the following error: Field 'id' doesn't have a default value"

2) I let Akeeba transfer the content. I deleted the existing database, re-created it and gave the proper user full permissions then restored using kickstart8_0

nicholas
Akeeba Staff
Manager

I have a really bad feeling about this. Can you try creating a new article in the site you backed up from and tell me if you get the same error message?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

KenHorse

I will have to restore to my development server. I am aware of the time difference between us (I am -7 UTC) so I will post the results later today. I will check your response tomorrow. Again, thank you for all your help with this

KenHorse

OK!  I was able to restore earlier than I first thought and the problem does NOT exist on my development server (MariaDB 10.11.6) running Debian12. And Akeeba backup also works properly.

The production server is MySQL 5.7.23-23 and 4.19.286-203.ELK.el7.x86_64

Must be a database (or kernel) version issue?

nicholas
Akeeba Staff
Manager

Kernel, no. Database version, tangentially yes.

Given our discussion so far, I am wondering how you managed to restore the site on a MySQL 5.7 server which does not support InnoDB as this would have been impossible. The root cause probably lies in there.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

KenHorse

This may be a moot point as the current hosting service won't provide an updated mySQL/MariaDB so I think we'll change hosting (as our contract with the current one is up in 2 weeks anyway). I do have a Joomla 5 version of the website running just fine on my development server

nicholas
Akeeba Staff
Manager

I think that's the best approach for many reasons. So, I guess, we're leaving this ticket open as likely resolved by future action?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

KenHorse

I would say it is resolved, yes. I really do appreciate all your effort in helping me with this

nicholas
Akeeba Staff
Manager

You're welcome!

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!